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SUMMARY 


PROSLIM 

Prototype  computer-based,  multimedia,  individualized  training 
systems  have  demonstrated  that  the  concept  of  computer-assisted  instruc¬ 
tion/  computer-managed  instruction  (CAl/CMi)  is  valuable  and  is  directly 
applicable  to  an  operational  Air  Force  training  environment.  Sub¬ 
stantial  savings  in  training  times  and  more  efficient  utilization  of 
resources  have  been  demonstrated.  However,  the  prototype  systems  in¬ 
corporated  research  capabilities  which  were  not  essential  to  effective 
routine  support  of  the  majority  of  Air  Force  training.  The  purposes 
of  this  study  were  (a)  to  identify  those  CAI/CMI  functions  with  the 
greatest  potential  payoff  for  Air  Force  training  and  (b)  to  develop  a 
functional  specification  for  a  low-cost  coi'iputer-based  system. 

APPROACH 

The  applicability  of  end  requirements  for  CAI  and  CM1  in  various 
Air  Force  training  environments  were  determined  by  S'-rrcg ; u>j  potential 
user  personnel.  Computer  system  archi tectures  were  surveyed  to 
determine  candidate  systems  to  meet  the  functional  requirements  of  the 
low-cost  CAI/CMI  system.  Alternative  terminal  hardware  devices  and 
communication  systems  vvere  surveyed,  in  order  to  recommend  an  economical 
set  of  input/output  devices  and  communications  interface  to  meet  the 
functional  requirements  of  the  system.  Various  computer  programing 
languages  were  analyzed  to  identify  a  language  which  would  be  cost 
effective  for  cAI/CMi  programming.  The  CAI/CMI  applications  software 
area  was  surveyed,  to  determine  the  extent  to  which  existing  software 
might  meet  Air  Force  training  needs. 

The  initial  assumptions  used  to  set  the  scope  of  the  study  were  os 
follows:  the  computer,  terminals,  communications ,  programming  language, 
and  applications  software  must  be  capable  of 

1.  Supporting  500  students  per  shift,  in  1500  hours  of  instruc¬ 
tion  across  five  courses,  with  150  instructional  hours  of  CAI 
and  with  five  C!11  transactions  per  student  per  shift. 

2.  Providing  a  oroven  student  progress  management  approach. 

3.  Supporting  the  oroduction  and  evaluation  cf  instructional 
materials  for  on-line  and  off-line  use. 

4.  Providing  a  set  of  standard  reports  for  various  levels  of  Air 
Force  instructional  personnel,  with  flexible  capability  to 
retrieve  and  analyze  training  performance  data  for  special 
purpose  reports. 


RESULTS 


Air  Force  instructors  and  supervisory  personnel  identified  a  broad 
range  of  functional  capabilities  in  the  CAI  and  CMI  areas  with  potential 
payoff  for  training.  A  total  of  167  functions  from  seven  major  CAI /CMI 
functional  a: eas  are  recomnended  for  inclusion  in  the  Air  Force  low- 
cost  system. 

A  survey  of  central  processing  units  identified  a  minicomputer  con¬ 
figuration  as  the  principal  candidate  for  meeting  the  functional  require¬ 
ments  of  a  dedicated  local  system.  t The  concept  of  stand-alone  devices 
was  considered  briefly.  It  became  obvious  that  the  myriad  of  combi¬ 
nations  could  not  be  dealt  with  within  the  scope  and  time  frame  of  the 
study. 

A  survey  of  terminals  and  communications  resulted  in  recommenda¬ 
tions  for  a  candidate  low-cost  student/administrative  terminal,  a  low 
resolution  graphics  terminal  with  color  CAI  capability,  a  high 
resolution  graphics  terminal  for  CAI,  and  a  management  terminal  reader/ 
printer  configuration. 

The  survey  of  applications  software  indicated  that  all  of  the 
recommended  functions  have  been  implemented  on  one  or  more  existing 
computer-based  instruction  systems,  demonstrating  the  technical 
feasibility  of  implementing  the  recomnended  functions.  At  the  present 
time,  various  computer-based  systems  present  more  or  less  of  the 
recommended  functions.  Some  of  these  existing  systems,  howe'er,  are 
implemented  on  large  centralized  mainframes  capable  of  supporting 
satellite  operations,  and  incorporate  sophisticated  research  and 
development  capabilities  that  are  not  essential  to  routine  support  of 
the  majority  of  Air  Force  training. 
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1.0  INTRODUCTION 


Prototype  computer-based  individualized  training  systems  have 
been  developed  and  implemented  at  a  number  of  Department  of  Defense 
( DOD)  and  civilian  installations.  One  such  training  system  is  the 
Advanced  Instructional  System  (AIS)  at  Lowry  AFB.  the  results  of  a 
variety  of  research  and  development  (R&D)  projects  indicate  that 

1.  Application  of  computer-based  instruction  (CBI)  resulted  in 
substantial  savings  in  training  time  and  more  efficient 
utilization  of  resources. 

2.  An  operational  training  system  does  not  need  sophisticated 
research  capabilities  which,  although  incorporated  into  pro¬ 
totype  systems  and  required  for  research,  are  not  essential 
to  effective  support  of  the  majority  of  resident  technical 
training  operational  requirements. 

3.  Full  and  economical  exploitation  of  the  demonstrated  oper¬ 
ational  capabilities  of  the  prototype  systems  in  routine 
support  of  resident  training  requires  additional  application 
and  system  refinement. 

Innovations  in  computer  technology  have  drastically  reduced  the 
prices  of  computer  components  and  terminals.  Consequently,  the  goals  of 
computer-based  training  should  be  reexamined  in  the  context  of  1979 
state-of-the-art  computer  technology. 

The  present  study  establishes  the  requirements  for  a  purely  oper¬ 
ational  system  and  integrates  those  requirements  into  a  system  design 
based  on  the  best  of  current  computer  technology.  The  principal 
objective  was  to  design  a  cost-effective  system  configuration  which 
would  efficiently  support  the  CBI  functions  identified  as  having 
potential  payoff  from  an  operational  viewpoint,  without  including  "nice 
to  have"  or  "rich"  capabilities  in  support  of  RSD  functions.  The 
products  of  this  study  are  as  follows: 

1.  An  overall  system  specification,  delineating  a  set  of 
functional  requirements  for  a  low  cost  computer-assisted  in¬ 
struction/computer-managed  instruction  (CAI/CMI)  system 
(Appendix  H). 

2.  This  Technical  Report,  presenting  the  assumptions,  data 
collection  methods,  analyses,  and  results  related  to  CAI  and 
CM I  system  capabilities,  selected  system  architectures, 
communication  systems,  terminal  hardware,  and  programming 
languages  and  software.  This  report  includes  the  conf igurat ion 
of  an  example  low-cost  CAI/CMI  system  with  communications 
equipment,  terminal  hardware,  applications  programs  software, 
and  a  language  which  will  satisfy  the  operational  requirements 
within  established  constraints  and  assumptions. 
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2.0  STUDY  PLAN 


The  research  plan  called  for  accomplishment  of  the  following  major 
tasks: 

1.  Determine  Functional  Requirements  of  Users  -  survey  Air  Force 
users,  analyze  survey  data,  and  determine  functional  require¬ 
ments  for  a  low-cost  CAI/CMI  system  for  operational  use  in  Air 
Force  resident  technical  training  programs. 

C.  Survey  Hardware,  Software,  and  Systems  -  survey  available  hard¬ 
ware,  software,  and  languages,  and  identify  candidates  for 
central  processor  units  (CPUs),  conmuni cat  ions .  terminals, 
software,  and  languages. 

a.  Document  and  Design  -  write  a  functional  specification  and  con¬ 
figure  an  example  of  a  state-of-the-art  hardware/ 
conimini cations/ terminal  s/software  system  to  meet  the  functional 
requirements. 

Figure  1  illustrates  the  task  flow  for  this  research  plan.  The 
following  paragraphs  describe  procedures  for  accomplishing  the  task?. 

In  order  to  provide  a  framework  around  which  the  design  work  ot 
this  study  could  proceed,  some  assumptions  were  made  regarding  the  in¬ 
tended  uses  and  environment  for  the  low-cost  CAI/CMI  System.  Those 
assumptions  are  as  follows: 

1.  System  design  will  be  oriented  toward  resident  technical  train¬ 
ing. 

2.  System  design  will  be  oriented  toward  a  dedicated  local  system, 
rather  than  a  large  centralized  mainframe-oriented  system  or 
stand-alone  minisystems.  A  modular  approach  to  expansion  will 
be  incorporated,  to  facilitate  tradeoffs  such  as  fewer  on-line 
students  in  exchange  for  heavier  CA1  usage.  The  effects  of 
increased  loads  on  memory  and  mass  storage  requirements  will 

be  considered.  ‘ 

i 

J.  System  design  will  bo  targeted  toward  providing  operational 
CAI/CMI  support  for  approximately  500  students  per  shift.  It 
is  also  assumed  that:  the  500  students  are  distributed  across 
five  different  courses;  the  five  courses  account  for  1,500 
hours  of  instruction;  each  student  averages  five  CMI  trans¬ 
actions  per  shift;  not  more  than  10'  of  the  students  are  using 
CAl  at  any  one  time;  not  more  than  nine  administrators  (in¬ 
structors  or  supervisory  personnel)  are  on-line  simultaneously; 
and  batch  processing  is  accomplished  off-shift  (i.e.,  during 
periods  of  low  or  no  student  CAl  CMI  load). 
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3.0  FUNCTIONAL  REQUIREMENTS  OF  USERS 


The  initial  step  toward  designing  a  low-cost  CAI/CMI  system  was 
to  determine  those  computer- provided  functions  which  would  be  of  value 
in  Air  Force  training.  To  this  end,  the  following  tasks  were  accom¬ 
plished: 

1.  Preparation  of  a  listing  of  CA!  and  CMI  functions  which  could 
be  provided  by  a  computer-based  training  system. 

2.  Preparation  of  a  rating  scale  with  which  Air  Force  users  could 
estimate  payoff  potentials  for  various  CAI/CHI  functions. 

3.  Identification  of  a  suitable  sample  of  potential  Air  Force 
users  of  a  low-cost  CAI/CMI  system. 

4.  Collection  of  data  from  the  sample  of  potential  users. 

5.  Analysis  of  the  data. 

6.  Determination  of  the  CAI  and  CMI  functions  which  should  be  pro¬ 
vided  by  the  low-cost  system. 

3.1  LISTING  OF  CAI/CMI  FUNCTIONS 

A  comprehensive  listing  of  184  potential  CAI  and  CMI  functions  was 
developed,  describing  those  functions  which  could  be  incorporated  into 
a  CBI  system.  This  listing  was  Intended  to  include  any  technically 
feasible  functions  which  could  be  provided  for  Air  Force  training  by  a 
computer  based  system,  and  which  were  not  considered  to  be  R&D 
functions.  The  outcome  of  this  effort  was  a  listing  of  functions  in 
seven  major  areas:  Student  Testing  and  Evaluation,  Student  Assignment, 
Student  Progress  Management,  Support  for  Guidance  and  Counseling,  CAI, 
Production  and  Maintenance  of  CMI  Data  Base  and  CAI  Materials,  and  In¬ 
formation  and  Reports  (see  Appendix  A). 

The  Appendix  A  listing  of  the  184  functions  includes  the  numbering 
scheme  that  is  used  throughout  this  report  in  referring  to  individual 
CAI/CMI  functions— Roman  numerals  designate  the  seven  major  areas, 
followed  by  letters  and  Arabic  numbers  that  designate  subcategories  and 
individual  functions. 

3.2  PREPARE  RATING  SCALE  AND  SURVEY  FQ|?MS 

The  184  CAI/CMI  functions  were  incorporated  into  survey  forms, 
with  a  rating  scale  intended  for  use  by  Air  Force  training  and  manage¬ 
ment  personnel  in  estimating  the  relative  values  that  the  functions 
would  have  for  operational  training.  The  rating  scale  was  oriented 
toward  trie  concept  uf  "payoff  potential."  The  introduction  to  the 
survey  form  and  the  explanation  of  the  rating  scale  are  shown  in 
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Appendix  B.  The  explanation  points  out  thet  payoff  potential  for  Air 
Force  training  can  come  fro*  any  of  the  following: 

0  Reduced  training  time  -  students  get  through  faster. 

0  Reduced  administrative  load  on  Instructors. 

0  More  efficient  use  of  training  resources. 

0  Improved  quality  control  over  students  and/or  courseware. 

c  Improved  mnnagccwnt/admlnlstratlon  of  training. 

°  Improved  courseware  development  and  evaluation  processes, 

resulting  In  better  Instructional  materials  and  reduced  develop¬ 
ment  times. 

The  Instructions  for  using  the  rating  scale  (Appendix  B)  asked 
participants  to  consider  how  much  each  of  the  functions  would  be  worth 
to  Air  force  training  within  the  context  of  the  participant's  duty 
assignment,  ami  to  assign  ratings  using  the  following  scale: 

1  -  Very  high  payoff  potential. 

-  High  payoff  potential. 

.1  -  Moderate  payoff  iwtcntial. 

4  -  Low  payoff  potential. 

5  "  No  payoff  potential. 

\  -  No  opinion  -  this  function's  effects  would  be  outside  my  area, 
and  l  cannot,  estimate  t ts  payoff  potential. 

Because  of  the  large  number  of  CAI/CMI  functions  Included  In  the 
listing,  two  survey  forms  were  prepared.  One  form  Included  the 
functions  that  were  of  potential  value  in  the  conduct  of  resident 
technical  training  at  the  Instructor  level.  The  second  form  Included 
the  functions  that  were  of  potential  value  for  Air  force  managers  and 
supervisors--!'. g. ,  course  supervisors,  branch  chiefs,  and  training 
advisors . 


The  instructor  survey  form  asked  for  ratings  on  the  following  items 
(see  Appendix  A): 

category  i  (Student  Testing  and  'valuation),  all  items  except 
item  IB. 

t'ateoory  11  (Student  Assignment),  all  items  except  item  UP. 


f 


11 


Category  III  (Student  Progress  Management) ,  all  Items. 

Category  IV  (Support  for  Guidance  and  Counseling),  all  items. 

Category  V  (Computer  Aided  Instruction),  all  items. 

Category  VI  (Production  and  Maintenance  of  CM3  Data  Base  ami  CAI 
Materials)  all  items  except  item  VIC. 

Category  VII  (Information  Retrieval  and  Reports),  all  items  listed 
under  VI IA*  Reports  for  Instructors. 

The  supervisor  survey  fonr.  asked  for  ratings  on  the  following  items 
(see  Appendix  A): 

Category  I  (Student  testing  and  Evaluation),  items  IA,  IB,  1C, 
and  1 1). 

Category  II  (Student  Assignment),  items  1 1 A .  113.  11C,  111),  HE, 
and  I  IF, 

Category  III  (Student  Proqress  Management),  items  II IA,  1113, 
1HB3-5,  and  UIC. 

Category  IV  (Support  for  Guidance  and  Counseling),  items  IVA  and 
IVB. 

Category  V  (Computer  Assisted  Instruction),  items  VA,  VB1-?,  and 
VC. 

Category  VI  (Production  and  Maintenance  of  CMl  Data  Base  and  CAl 
Materials),  Items  V1A1-?.  VIB.  VIC,  VlPl-6.  VIC.  VlCl-b  and 
VIF. 

Category  VI l  (Information  Retrieval  and  Reports),  all  items. 

Quantities  of  the  survey  forms  were  reproduced  for  use  in  the  data 
collection  effort.  Participants  in  the  survey  marked  their  responses 
directly  on  their  copies  of  the  forms,  and  identified  themselves  by  duty 
position.  Names  or  other  specific  identification  were  not  required. 

3.3  SAMPLING  OF  POTENTIAL  USERS 

Although  the  design  of  the  low-?bst  system  was  to  bo  oriented 
principally  toward  the  needs  of  resident  technical  training,  the  needs 
of  other  Air  Force  training  programs  were  also  to  be  identified.  This 
would  permit  documenting  the  extent  to  which  the  low-cost  system  would 
meet  the  needs  of,  for  example,  on-the-job  training  (OUT),  navigator 
training,  and  field  training  detachments  (FTOs).  Therefore,  samplings 
of  potential  users  from  resident  technical  training  and  from  a  number 
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of  other  Air  Force  training  programs  were  identified  for  inclusion  in 
the  survey  of  user  requirements. 

3.3.1  Resident  Technical  Training  Sample 

The  sample  of  resident  technical  training  personnel  was 
drawn  from  Air  Training  Conmand  (ATC)  Headquarters  at  Randolph  AFB  and 
from  four  Air  Force  Technical  Training  Centers  (TTCs) — Sheppard, 
Keesler,  Chanute,  and  Lowry. 

During  the  AIS  development  contract,  courses  at  the  TTCs  were 
analyzed  to  identify  those  which  were  prime  candidates  for  inclusion 
in  any  expansion  of  the  Air  Force's  CBI  effort.  Based  on  that  earlier 
analysis,  the  following  courses  were  included  in  this  survey  of  user 
requirements: 

Sheppard  TTC 

Air  Cargo  Specialist  (3BRB60531) 

Disbursement  Accounting  Specialist  (3ABR67232) 

Faculty  Development  (3A1R75100-4) 

Keesler  TTC 

Personnel  Specialist  (3ABR73230) 

Ground  Radio  Equipment  Mechanic  (3ARR30434) 

Telecommunication  System  Control  Specialist  (3A3R30730) 
Taculty  Development  (3AIR75100-4) 

Chanute  TTC 

Aerospace  Ground  Equipment  Repairman  (3ABR42335) 

Life  Support  Specialist  (3A3R92230) 

Airframe  Repair  Specialist  (3ARR42735) 

Lowry  TTC 

Munitions  Maintenance  Specialist  (3ABR46130) 

Inventory  Management  Supervisor  (3ABR64S70-1 ) 

Faculty  Development  (3AIR75100-4) . 

Personnel  from  several  other  ATC  resident  technical  training  pro¬ 
grams  wore  included  in  the  user  survey  because  of  their  involvement 
with,  and  knowledge  of.  Air  Force  computer-based  training: 

Sheppard  TTC  (users  of  PLATO) 
i'ealth  Care  Sciences 


Chanute  TTC  (users  of  PLATO) 

Vehicle  Repairman  (3ABR47231,  3ABR47232) 

Aircraft  Pneudraulics  (3ABR42334) 

Plans  and  Requirements  Division 
PLATO  Development  Group 

Lowry  TTC  (users  of  AIS) 

Inventory  Management  Specialist  (3ABR64530) 

Materiel  Facilities  Specialist  (3ABR64G31) 

Precision  Measuring  Equipment  Specialist  (3ABR32430) 

Weapons  Mechanic  (3ASR46230) . ' 

The  Air  Force  memoranda  requesting  the  TTCs'  participation  in  the 
survey  of  user  needs  requested  "...cooperation  in  the  Feasibility  Study 
Survey  of  the  following  individuals/agencies  involved  with  conduct  of 
the  above  courses: 

1.  The  Deputy  Commander  for  Training 

2.  The  Training  Advisor 

3.  The  Technical  Training  Group  Commander(s) 

4.  The  Chief,  Plans/Requirements  Division 

B.  The  Chief,  Branch  Curriculum  Development  Unit 
-Resident  Course  Representatives 
-Nonresident  Course  Representatives 
-Instructor  and  Course  Evaluation  (ISM)  Representatives 

6.  Course  Supervisors  of  the  above  Courses 

7.  Ten  Instructors  from  each  of  the  above  Courses." 

The  participation  of  personnel  from  Technical  Training  (TT)  and 
Plans  (XP),  HQ  ATC  Randolph  AF8,  was  also  requested.  These  staff 
agencies  are  responsible  for  planning,  progranninc,  administering,  and 
managing  ATC  technical  training  at  the  ATC  Headquarters  level. 

3.3.2  Other  Air  Force  Training  Sample 

The  following  instructors  and  supervisors  from  training 
programs  other  than  resident  technical  training  were  identified  for  in¬ 
clusion  in  the  survey  of  user  requirements: 

1.  FTu/OJT  instructors  and  supervisors  from  Sheppard,  Mather,  and 
Oavis  Monthan  AFBs. 

2.  Program  managers  and  staff  personnel  from  Air  University. 

3.  Staff  personnel  from  HQ  ATC  (DO),  Randolph  AFB  (pilot  training). 
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4.  Instructors  and  staff  personnel  from  Undergraduate  Navigator 
Training  (UNT),  Mather  AFB. 

3.4  DATA  COLLECTION 

The  necessary  survey  data  were  collected  at  each  of  the  sites 
identified  as  part  of  the  user  survey  sampling.  The  preferred  pro¬ 
cedure,  observed  wherever  the  participating  personnel  could  accommodate 
the  demands  on  their  time,  was  as  follows: 

1.  A  group  of  Air  Force  participants  gathered  for  a  30-ntnute 
briefing  which  explained  the  purposes  of  the  survey  and  the 
meanings  of  CAI,  CMI,  and  other  special  terms  used  in  the 
survey  foms. 

2.  Survey  forms  were  distributed  to  the  participants. 

3.  Participants  completed  the  survey  fons. 

4.  The  survey  forms  were  collected. 

The  total  time  required  for  this  procedu; e  averaged  approximately 
2  hours.  Personnel  who  could  not  stay  for  the  entire  session  were 
permitted  to  take  their  survey  forms  with  them  tc  be  completed  later. 

In  those  cases,  a  representative  was  designated  to  collect  the  forms. 
Some  Lowry  TTC  personnel  were  already  familiar  with  AIS  computer-based 
training,  and  the  briefing  was  omitted.  Using  these  procedures,  the 
rate  of  return  for  survey  forms  was  better  than  96". 

For  some  training  program,  other  than  resident  technical  training, 
the  survey  form  were  not  deemed  approoriate.  In  these  cases,  the 
participants  were  Interviewed  to  determine  their  particular  functional 
requirements  for  a  conputer-b.ised  traininc  system . 

3.5  DATA  HALYSiS  PKCCED.JFE3 

The  r  os  per,  ss  data— ratings  of  'Myoff  p’tont-al  for  »he  CAI/C'tl 
functions  extracted  fror  the  ccuplrtoJ  survey  foro-s— *«cr?  entoroc  i **tn 
the  AIT  computer.  Then,  for  each  *‘t«’  ir.  the  ’1-  t;ng  of  functions, 
mean  payoff  ratings  and  distributions  of  ratings  by  groupinjs  of  re¬ 
spondents  wore  derived.  T.e  grouping  of  principal  interest  for  the 
design  phase  of  tv. i s  study  was  "all  ATC  resiie.it  technical  training 
respondents."  Groupings  of  ATC  respondents  inv'lved  with  training  pro¬ 
grams  ether  than  resident  technical  training  were  of  interest  ir. 
detraining  tie  extent  to  which  the  low-cost  system  would  be  able  tj 
weet  the  needs  of  other  types  of  Air  Force  training. 

The  'aricus  groups  included  in  the  survey  differed  in  size, 
ringing  f»\>n  1  to  23  participants  per  group.  It  was  assumed  t'm  all 
groups  were  of  equal  importance  and  should  contribute  equally  2c  me 
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nvuft  payoff  ratings.  Therefore,  means  were  derived  for  each  function 
by  averaging  the  reans  fror,  each  group  entering  into  tin?  analysis. 

Thus,  the  size  of  a  grou.*  did  not  oetercine  that  group's  contribution 
to  the  overall  nean  ratings. 

3.5  OF.  PrlMI  NATION  OF  T  NOTIONAL  ncOUHtfCITS 

The  user  ratings  of  the  CAI/CMI  furctior.s  and  the  sys tc~  impacts 
of  chose  functions  were  two  priir>?  consi derations  in  determining  the 
f  motional  require lents  for  the  low-cost  system.  The  end  product  was 
a  listing  of  CAI/CMI  functional  requirements  plus  e  rc-cot  nendat  ion  for 
each  'unction  that  1c  Iv  either  retained  as  a  function  to  he  provide, 
by  the  lcw-cost  systcr,  or  eliminated  fro.i  r  .rther  consideration. 

d.t.l  Results  From  Res : dent  ’echnice:  Trj’ni .r, 

A  total  of  personnel  involved  in  resident  technical 
training  completed  forms  for  the  survey  cf  j>er  functional  requirements. 
The  listr-tuiion  of  participants  is  shown  in  Tibi*1  1. 

The  resjUs  fr;n  t:u*  survey  of  ATC  resident  technical  trainin' 
orogra. is  are  shewn  in  Appendixes  C  and  r>.  Ap^ndix  C  lists  the 
f  motions  In  the  order  in  which  they  appeared  on  the  Svrvcy  for, is  and 
indicates  the  mean  payoff  ratings  am.  tr.e  system  impacts  (file,  needed 
and  estimated  CPU  ..sage)  for  each  of  the  functions.  fpi>endi*  C  lists 
the  functions  in  order  of  average  estinwted  payoff,  and  includes  the 
distribution  of  ratings  given  to  the  functions  by  the  survey  >arttci- 
pancs. 

The  user  siiVvcy  results  can  *>e  summarized  as  fellows: 

1.  Men  racings  of  the  134  functions  range  frw  the  highest- 
’’ated  item  (flag  Test  Itrn  Alternatives  Missco  by  More  Than 
70t  of  Students)  with  *.  tear  ratino  of  2.07,  to  the  lowest- 
rated  item  (Students  Select  retired  Alternate  Module  Assign¬ 
ments)  with  a  near,  rati  .y  of  3.71. 

t.  Of  the  131  listed  functions,  M  received  average  ratings 
better  than  7.50,  and  115  received  average  ratings  less  than 
2.50  but  better  less  3. CO.  Only  3  of  the  functions  rated 
worse  than  3.00  (moderate  oavoff). 

3.  Eighty-twc  of  the  functions  were  rated  1  (very  high  payoff)  or 
2  (high  payoff)  by  50*;  or  more  of  the  2t>3  participants. 

4.  Even  the  lowest-rated  functions  (Students  Select  Alternatives, 
and  Heuristics  Rules  For  Assignment  Selection)  were  rated  high 
or  very  high  payoff  by  move  than  7r  of  the  participants. 
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TABLE  1.  RESIDENT  TECHNICAL  TRAINING  PROGRAMS  AND 
NUMBERS  OF  PEOPLE  SURVEYED 

LOCATION/ PROGRAM _ INSTRUCTORS  SUPERVISORS 

SHEPPARD  TTC 


Air  Cargo  Specialist 

9 

8 

Disburses  oit  Accounting 

3 

5 

Faculty  Development 

11 

10 

Health  Care  Sciences 

9 

16 

Total 

KEESLER  TTC 

Ground  Radio  Repairman 

7 

Personnel  Specia'ist 

9 

6 

Faculty  Development 

- 

4 

Telecommunication  Specialist 

5 

4 

Total 

CHANUTE  TTC 

AGE  Repairman 

8 

15 

Vehicle  Repairman 

15 

3 

Life  Support  Specialist 

4 

7 

Aircraft  Pneudraulics 

10 

3 

Airframe  Repair  Specialist 

5 

4 

Plans  and  Requirements  Div. 

- 

6 

PLATO  Development  Group 

- 

4 

Total 

LOWRY  TTC 

Inventory  Management 

9 

1 

Materiel  Facilities 

6 

- 

Precision  Measurement  Equip. 

15 

5 

Inventory  Mgt.  Specialist 

5 

- 

Munitions  Mamt.  Specialist 

- 

4 

Weapons  Mechanic 

5 

4 

Faculty  Development 

3 

6 

Total 

RANDOLPH  ALB  (HQ  ATC) 

Technical  Training  (TT) 

• 

11 

Plans  (XP) 

- 

3 

Total 


TOTALS 


1 38  1 30 

17 


TOTALS 


65 


43 


04 


63 


14 


360 


5.  The  highest-rated  functions  (Flag  Alternatives  Missed  By  More 
Than  70';.),  was  rated  no  payoff  by  only  one  of  the  participants. 

6.  The  numbers  of  X  (don't  know)  responses  ranged  from  A  (for  On 
Line  Test  capabilities)  to  FT  (for  CA1  Authoring  Editor 
Automatically  Structures  Sequencing  Of  Frames). 

Mean  ratings  and  correlations  between  ratings  from  the  various 
resident  training  organizations  involved  are  shown  in  Table  2.  The 
overall  mean  ratings  range  from  2.91  (slightly  better  than  moderate 
payoff  average  for  the  184  functions)  at  Lowry  up  to  2.48  (midway 
between  moderate  and  high  payoff)  at  Keesler.  The  correlations  are  all 
positive  and  range  from  0.30  for  Lowry/Randolph  up  to  0.78  for  Sheppard/ 
Keesler.  There  is  moderate  agreement  between  organizations  in  their 
ratines  of  the  functions  and  quite  close  correspondence  between  organ¬ 
izations  in  their  overall  mean  ratings.  In  other  words,  the  organ¬ 
izations  showed  moderate  agreement  as  to  which  functions  were  most,  and 
least,  important,  and  agreed  rather  closely  as  to  the  potential  payoff 
of  CA I /CM I  for  their  particular  training  operations. 

TARlF  2.  MEAN  RATINGS  FROM  RESIDENT  TRAINING  ORGANIZATIONS 
AND  CORRELATIONS  BCTWECtl  ORGANIZATIONS 


L0CA1 ION 

MFAN 

CORRELATIONS 

RATING  KEESLER 

CHANUTE 

LOWRY 

RANDOLPH 

SHEPPARD 

2.50  .78 

.57 

.41 

.39 

KEESLCR 

2.48 

.43 

.36 

.44 

CHANUTE 

2.62 

.63 

.33 

LOWRY 

2.91 

.30 

RANDOLPH 

2.78 

Mean  ratings  from  personnel  with 

and  without  CBI  experience  at 

Lowry  and  at  Chanute  are  shown  in  Table  3.  These  comparisons  are  of 
interest  in  determining  if  experience  with  CBI  results  in  any  overall 
cnange  in  how  the  payoffs  of  CAI/CMI  functions  are  viewed.  At  Lowry 
TTC,  the  average  rating  from  personnel  in  AIS  copses  was  2.70,  some¬ 
what  better  than  the  3.13  average  for  personnel  in  the  non-PLATO 
courses.  At  Chanute,  the  overall  mean  rating  from  personnel  in  PLATO 
courses  was  2.37,  somewhat  better  than  the  overall  mean  of  2.58  from 
personnel  in  the  non-PLATO  courses.  It  appears  that  experience  with  C81 
did  not  markedly  affect  the  mean  ratings  of  CAI/CMI  functions. 


r 


5 


TABLE  3.  MEAN  RATINGS  FROM  PERSONNEL  ASSOCIATED  WITH 
COMPUTER-BASED  AND  NON-COMPUTER-BASED  COURSES 

WRY  TTC 


AIS  COURSES 

MEAN 

NON-AIS  COURSES 

MEAN 

ventory  Management 

2. S3 

Inventory  Mgt.  Supvr. 

3.46 

terlel  Facilities 

2.51 

Munitions  Maintenance 

3.25 

ecis.  Meas.  Equip. 

2.89 

Faculty  Development 

2.69 

apons  Mechanic 

2.52 

Overall  Means 

2.70 

3.13 

IANUTE  TTC 

• 

PLATO  COURSES 

MEAN 

NON-PLATO  COURSES 

MEAN 

ihicle  Repairman 

3.06 

AGE  Repairman 

2.45 

leudraul ics 

2.63 

Life  Support 

2.53 

.ATO  Devel .  Group 

1.41 

Airframe  Repair 

2.70 

Overall  Means 

C.  O/ 

2.58 

The  last  item  on  the  survey  forms  asked  participants  to  list  any 
dditional  functions  that  should  be  provided  by  a  computer-based  traili¬ 
ng  system.  There  were  57  participants  who  responded  to  this 
jestion.  Most  of  the  responses  added  more  detail  to  already-listed 
unctions,  or  commented  (positively  or  negatively)  on  the  survey  and  on 
31.  The  substantive  suggestions  for  additional  functions,  and  the  sub- 
equent  actions  taken,  a-e  as  follows: 

1.  System  to  adjust  the  honor  graduate  score  to  keep  within  the 
10°^  ATC  requirement  -  Action:  to  be  included  in  the  low-cost 
system  specif  icatim. 

2.  System  to  provide  a  data  base  to  handle  credit  in  the 
Community  College  of  the  Air  Force  (CCAF)  for  courses 
completed  -  Action:  not  to  be  included  in  low-cost  system, 
because  of  potential  interface  problems,  and  because  the 
low-cost  system  is  to  be  a  local  system. 

3.  System  to  provide  dial-up  conversational  capability  between 
terminals,  for  student-instructor  interactions  -  Action:  not 
to  be  included  in  low-cost  system,  because  of  likelihood  of 
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misuse  and  because  the  initial  low-cost  system  is  oriented 
toward  local  resident  technical  training  use,  rather  than 
terminals  distributed  at  remote  locations. 

4.  System  to  assign  instructors  automatically,  within  the  course, 
based  on  qual i f ications,  availability,  etc.  -  Action:  not  to 
be  included  because  of  unfavorable  institutional  chanoe 
aspects  and  overlap  with  other  Air  Force  personnel  management 
systems . 

b.  System  to  assiqn  graduating  students  to  duty  stations,  based 
on  student  proficiencies/def iciencies  and  on  duty  station 
requirements  -  Action:  not  to  be  included,  because  of  unfavor¬ 
able  institutional  change  aspects  and  overlap  wi  tti  other 
personnel  management  systems. 

Examination  of  the  survey  results  leads  to  the  conclusion  that  all 
f  the  lAl/CMI  functions  are  of  some  potential  value  to  resident 
ecnnical  training.  Furthermore,  estimates  of  payoff  potential  range 
long  a  continuum,  without  clear  separation  points,  from  low  payoff  up 
o  high  payoff.  The  moderate  correlations  between  organizations  suggest 
iverse  training  needs  --  i.e.,  the  subset  of  functions  with  the  great- 
st  payoff  potential  for  one  ATC  organization  is  not  the  subset  with 
reatest  potential  for  another  organization.  The  system  impact 
stimates  shown  in  Appendix  C  indicate  that  file  requirements  and  CPU 
imos  are  determined  principally  by  inclusion  or  omission  of  major 
iroups  of  functions,  not  by  inclusion  or  omission  of  a  few  functions 
rom  within  a  major  group.  Therefore,  even  though  specific  functional 
equi renents  differ  from  program  to  program,  it  may  be  most  practical 
o  provide  a  system  which  has  all  desired  r.apabi  1  i ti es  and  allow  the 
irograms  to  use  those  functions  they  want  and  ignore  others. 

As  the  final  step  in  the  analysis  of  functional  requirements  for 
^TC  resident  technical  training,  A1S  experience  and  lessons  learned 
/ere  applied  in  determining  the  final  list  of  CAI/CMI  functions  to 
,‘nter  into  the  system  analysis  for  the  low-cost  system.  The  functions 
: ha t  were  dropped  from  further  consideration,  and  the  rationales  for 
imitting  these  functions  from  the  low-cost  system,  are  listed  in  Table 
1.  The  retained  functions  enter  into  the  system  analysis  and  cesign 
ihase  of  this  study. 
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TABLE  4.  FUNCTIONS  OMITTED  FROM  THE  LOW-COST 
SYSTEM  SPECIFICATION,  AND  RATIONALES  FOR  OMISSIONS 

FUNCTION  (Complete  Statements  are  in  Appendix  A)/RATIONALE  FOR  OMISSION 


i 


1.  IA2 


2.  IA3 


3.  IA6 


4.  IB4 


b.  IB6 

PERFORMANCE/ 

Too  difficult  for  authors  to  define  the  adaptation  to  be 
used.  Requires  data,  and  analyses,  to  relate  student 
perfornance  on  other  lessons  in  the  course  to  "what  should 
be  done  at  this  step  in  this^test?"  The  necessary  analyses 
and  interpretations  are,  practically,  not  cost  effective 
in  ATC  resident  technical  training 

6.  I  IBS  STUDENT  ASSIGNMENT-ORDER  SELECTED  EY  INSTRUCTOR/ 

This  capability,  if  provided,  would  impose  on  instructors 
the  task  of  determining  the  sequence  in  which  each  student 
should  take  the  course  lessens.  It  is  more  practical  to 


STUDENT  TESTING  AND  EVALUATION  -  CORRECT  SCORE  FOR  GUESSING/ 
Rated  low  in  survey  of  Air  Force  needs,  not  viewed  as 
educationally  important  by  ATC  personnel. 

STUDENT  TESTING  AND  EVALUATION-WEIGHT  QUESTIONS  DIFFERENTIALLY/ 
Not  practical,  in  Air  Force  courses,  to  determine  the  weight¬ 
ings.  Requires  subjective  judgements  from  subject  matter 
experts,  or  requires  a  large  amount  of  performance  data 
collected  within  the  course  and  from  the  field,  with  ti mo- 
consuming  data  analyses  and  interpretations  to  relate 
responses  with  field  performance.  The  demand  on  course 
personnel  to  accomplish  these  tasks  is  not  worthwhile,  in 
terms  of  educational  benefits. 

STUDENT  TESTING  AND  EVALUATION-PASS  SPECIFIC  QUESTIONS/ 

Rated  low  by  ATC  personnel ,  viewed  as  low  payoff.  Determin¬ 
ing  critical  "must  answer"  questions  requires  subjective 
judgements  by  course  personnel,  supplemented  with  data 
collection  and  analyses  to  validate  the  decisions.  The  ed¬ 
ucational  returns  will  not  justify  the  demands  on  ATC  course 
personnel . 

STUDENT  TESTING  AND  EVALUATION-ON-LINE  CONSTRUCTED  RESPONSES/ 
The  costs  of  software  development  for  this  capability,  and 
the  demands  made  on  authors,  outweigh  the  educational  gains 
that  can  be  expected  in  ATC  resident  technical  training. 
Authors  are  required  to  specify,  in  great  detail,  allowable 
alternate  spellings,  alternate  orderings  of  words,  etc. 

Only  after  careful  observations  of  student  responses  can  an 
author  determine  that  the  correct  answers  are  all  treated  as 
correct  and  that  all  the  incorrect  answers  are  scored  as 
incorrect. 

STUDENT  TESTING  AND  EVALUATION  ON-LINE,  ADAPTIVE  CN  COURSE 
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TABLE  4.  (Continued) 


have  the  system  make  assignments  based  on  criteria  deter¬ 
mined  by  the  Branch,  with  instructors  overriding  for 
exceptional  cases. 

7.  IICl  STUDENT  ASSIGNMENT- INSTRUCTOR  ASSIGNS  ALTERNATES/ 

As  for  the  preceding  item,  the  instructor  is  freed  of  routine 
if  the  system  makes  the  assignments  to  alternates,  with  the 
instructor  overriding  for  exceptional  cases. 

3.  IIC2  STUDENT  ASSIGNMENT-STijrZNT  SELECTS  DESIRE.')  ALTERNATES/ 

Rated  very  low  by  ATC  — instructional  payoff  is  believed  to  be 
quite  small.  Cmittirg  this  function  still  leaves  the  in¬ 
structors  free  to  override,  so  that  a  student  who  can  con¬ 
vince  an  instructor  to  intervene  may  take  any  desired 
al ternate. 

9.  1 1  IBS  STUDENT  PROGRESS  MANAGEMENT-GRADUATES:  SCHEDULE  OUT- 

PROCESSING/ 

The  system  will  flag  those  students  who  are  to  graduate 
during  some  designated  time  period.  However,  the  problems 
of  interfacing  with  all  of  the  organizations  involved  in  out- 
processing  students  is  outside  the  scope  of  the  low-cost 
system  effort. 

1C.  IIIC6  STUDENT  PROGRESS  MANAGEMENT-MANAGEMENT  TARGETS  RELATIVE  TO 
PLAN  OF  INSTRUCTION  LENGTH/ 

Average  course  length  is  more  immediately  related  to 
curriculum  changes  than  is  Plan  of  Instruction  length. 
Therefore,  management  functions  which  are  relative  to 
average  course  length  will  reflect  shortened  cr  lengthened 
curricula  sooner  than  if  relative  to  Plan  of  Instruction 
length. 

11.  VBA  COMPUTER-AIDED  INSTRUCTION  (CAI)-TOUCH  PANEL  OR  LIGHT  PEN/ 

Rated  low  by  ATC  personnel.  Seldom  needed  for  student  use. 
Principal  value  is  in  preparation  of  graphics  displays  for 
CAI  lessons.  This  will  be  accomplished  at  a  graphics 
terminal,  in  the  low-cost  system,  rather  than  at  a  student 
interactive  terminal. 

"V 

12.  VIIG1  INFORMATION  RETRIEVAL  AND  REPORTS-PRESERVICE  EDUCATION,  VOC. 

TRNG. ,  TO  MAKE  FORECASTS  OF  FltLD  PERFORMANCE/ 

Difficult  to  derive  predictions  based  on  pre-service 
vocational  education  data.  Good  predictions  would  require, 
for  each  course,  a  sufficient  data  base  from  which  to  derive 
reasonably  accurate  regression  equations  or  heuristic  rules, 
and  the  skill  and  time  to  perform  and  interpret  the  analyses. 
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TABLE  4.  (Concluded) 


13.  Y1IG2  INFORMATION  RETRIEVAL  AND  REPORTS-PRESERVlCE  OCCUPATION  AND 

JOB  EXPERIENCE  TO  MAKE  FORECASTS  OF  FIELD  PERFORMANCE/ 
Rationale  for  omission,  same  as  for  V1I61  above. 

14.  VIIG3  INFORMATION  RETRIEVAL  AND  REPORTS-AF  PERSONNEL  RECORDS  DATA 

(ASVAB,  ETC.)  TO  MAKE  FORECASTS  OF  FIELD  PERFORMANCE/ 
Rationale  for  omission,  same  as  for  VIIG1  above. 

15.  VIIG5  INFORMATION  RETRIEVAL  mND  REPORTS-SPECIALTY-RELATED 

DEFICIENCIES  TO  HAKE  FORECASTS  OF  FIELD  PERFORMANCE/ 
Rationale  for  omission,  same  as  for  V1IG1  above. 

16.  VIIG6  INFORMATION  RETRIEVAL  AND  REPORTS- PREDICT  STUDENTS' 

ADAPTABILITY,  PERFORMANCE,  AND  PROGRESSION  IN  OPERATIONAL 
FIELD  ASSIGNMENTS/ 

Rationale  for  omission,  sane  as  for  V1IG1  above. 

NOTE:  Of  the  VUG  set  of  functions  (Performance  Forecast  Reports  to  the 
Field),  VIIG4  will  be  provided  by  the  low-cost  system  -  Reports 
to  the  field  based  on  in-service  training  performance  data. 

17.  VI l Ml  SPECIAL  REPORTS-BASIC  DATA  RETRIEVAL,  BKGO/ 

Dropped  in  favor  of  the  higher-rated  Interactive 
Retrieval  With  Statistics  (VIIH3). 

18.  VIIN2  SPECIAL  REPORTS- INTERACTIVE  RETRIEVAL/ 

Dropped  in  favor  of  the  higher-rated  Interactive 
retrieval  With  Statistics  (VIIH3). 
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3.6.?  Results  from  Other  Air  Force  Training  Programs 

In  addition  to  surveying  ATC  resident  technical  training 
requirements,  other  Air  Force  training  programs  were  also  inch 'led. 

The  additional  programs,  and  numbers  of  participants,  are  listed  in 
Table  b.  As  described  earlier  (Section  3.4.1)  and  as  indicated  below, 
data  were  collected  using  survey  forms  and  by  interviews  in  which  the 
survey  form  served  as  a  guide. 


TABLE  5.  ADDITIONAL  PROGRAMS  1NCLUDF0  IN  SURVEY,  AND 
NUMBERS  OF  PARTICIPANTS 

INSTRUCTORS  SUPERVISORS 

FIELD  TRAINING  DETACHMENTS  &  ON- 

THE-JOB  TRAINING— SHEPPARD, - 

MATHER,  &  DAVIS  MONTHAN  AFBs 


Survey  Forms 

23 

9 

Interviews 

■) 

b 

AIR  UNIVERSITY 

Survey  Forms 

- 

11 

EXTENSION  COURSE  INSTITUTE 

Interviews 

- 

8 

HQ  ATC  (DO),  RANDOLPH  AFB 
(Pilot  Training)* 

Survey  Forms 

- 

1 

MATHER  AFB  (Undergraduate 

Navigator  Training) 

Survey  Forms 

2 

5 

Interviews 

6 

11 

TOTALS 

33 

60 

*  Explicit  definition  of  the  computer-based  functions  required  by 
Flight  Training  is  given  in  Data  Automation  Requirement  DAR  A78-2, 
Time  Related  Instructional  Management  for  Pilot  Training,  Air 
Training  Command,  29  June  1978. 
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Results  from  the  survey  forms  are  tabulated  in  Appendix  E.  The 
mean  ratings  and  the  correlations  between  ratings  from  the  various  pro¬ 
grams  are  shown  in  Table  6.  Resident  technical  training  is  also 
included  in  Table  6  for  purposes  of  comparison. 

TABLC  b.  MEAN  RATINGS  AND  CORRELATIONS  I  OR  VARIOUS 
AIR  FORCE  TRAINING  PROGRAMS 

OVERALL  CORRELATIONS 

MEAN 


PROGRAM* 

RATINGS 

UNT 

PT 

AU 

RLS 

OJT/I TO 

2.92 

.2b 

.04 

.34 

.37 

UNT 

2.68 

.73 

.08 

.13 

PI 

3.09 

.10 

-.08 

AU 

3.31 

.hi 

RLS 

2.0  2 

*  On-The-Job  Training/! i el <1  Training  Detachments  (OJT/ITD),  Under¬ 
graduate  Navigator  Training  (UNI),  Pilot.  1  raining  (PI),  and  Air 
University  (AU).  Resident  T'1  "finical  lraining  (IUS)  is  included  for 
comparison. 


The  overall  mean  of  the  ratings  from  resident  technica'  *■  aiding 
is  2.o2  --  between  High  (2.0)  and  Moderate  (J.O)  payoff.  "  io  mean 
rating  from  UNI  is  down  only  slightly  (2.btl),  followed  by  OJT/ETD 
(2.92),  PT  (3.09),  and  AU  (3.31).  Those  other  Air  I  orce  training  pro¬ 
grams  do  see  payoff  potential  in  the  CA1/CM1  functions,  but  not  as  much 
potential  as  in  resident  technical  training.  The  correlations  indicate 
some  conmonal i ty  among  the  needs  of  OJT/ITD,  All,  and  resident  technical 
training  (correlations  of  .34,  .37,  and  .hi).  There  is  also  consider¬ 
able  comnonality  between  navigator  training  and  pilot  training 
(correlation  of  .73). 

The  notes  from  the  interviews  were  analyzed,  and  the  results  are 
shown  in  Appendix  I.  Recause  the  survey  form  served  as  a  guide  for  the 
interviews,  the  respot  os  fall  into  the  seven  categories  used  on  the 
survey  form  but  do  not  correspond  exactly  to  the  wordings  used  in  the 
survey.  Each  function  listed  in  Appendix  I  was  categorized  as  either 
(a)  already  provided  by  the  functions  listed  on  the  survey  form,  (b) 
already  provided  by  the  survey  form  functions  but  requiring  minor 
modification  to  adapt  from  resident  technical  training  (for  example, 
referencing  tost  items  to  Career  Development  Course  volumes  instead  of 
to  resident  training  lessons,  or  (c)  not  provided  by  the  survey  form 
functions.  These  categories  are  also  indicated  in  Appendix  i. 
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4.0  HARDWARE  SURVEYS  AND  ANALYSES 


To  provide  a  framework  for  configuring  a  low-cost  system,  the 
following  training  scenario  and  design  goals  were  established: 

1.  Approximately  500  students  per  shift,  enrolled  in  five  courses. 

2.  Not  more  than  10%  of  the  students  using  CAI  at  any  one  time. 

3.  An  average  of  five  CMI  interactions  per  student  per  shift. 

4.  Not  more  than  ten  administrative  personnel  on-line  at  any 
one  time. 

5.  Productive  response  times  as  follows: 

a.  less  than  or  equal  to  1  second  average  response  time  for 
simple  operations. 

b.  less  than  or  equal  to  3  seconds  average  response  time  for 
more  complex  operations. 

Tne  training  scenario  and  design  goals  suggest  a  probable  price 
range  (1979)  of  $500,000  to  $600,000  for  mainframe,  peripherals, 
terminals,  and  associated  support  hardware. 

Surveys  of  hardware  were  restricted  to  selected  equipment  items  in 
the  areas  of  full  size  and  minicomputer  mainframes  and  peripherals,  data 
processing  equipment,  terminals,  and  related  communication  systems. 
Equipment  was  evaluated  as  to  its  potential  for  application  in  the  low- 
cost  system.  The  survey  methods  included  reviews  of  literature,  trips 
to  remote  sites  to  consider  classes  of  equipment  or  specific  hardware, 
and  visits  to  trade  shows  and  seminars.  The  DataPro  Research  Corp. 
(1973)  and  other  surveys  furnished  preliminary  reviews  of  equipment 
manufacturers 1  specifications  on  computer  mainframes  and  peripherals, 
terminals,  communications  systems,  and  software. 

4.1  CANDIDATE  CPU  ARCHITECTURES 

The  purpose  of  this  portion  of  the  study  was  to  analyze  existing 
computing  architectures  and  to  identify  one  which  would  provide  a  cost 
effective  approach  to  supporting  ATC  resident  technical  training  pro¬ 
grams.  To  accomplish  the  initial  investigation  tasks,  current  pub¬ 
lications  on  existing  computing  architectures  were  examined.  Then, 
considering  functional  capabilities  and  acquisition  costs,  a  number  of 
existing  computer  architectures  w.re  selected  as  an  initial  set  of  can¬ 
didate  CPUs  for  more  detailed  investigation.  Data  on  the  efficiencies 
of  each  of  the  systems  were  analyzed  to  determine  the  best  performing 
CPUs.  Then,  the  criteria  of  software  quality  and  reliability,  hardware 
reliability,  and  effectiveness  of  maintenance  service  were  considered 
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as  additional  evaluation  factors. 

Within  the  framework  of  the  training  scenario,  design  goals,  and 
evaluation  factors  described  above,  many  large-scale  and  super  computers 
could  fulfill  the  functional  requirements.  However,  the  CPU  and  input/ 
output  (I/O)  power  they  provide  are  not  necessary  in  this  application, 
and  the  costs  of  such  machines  are  above  the  expected  price  range.  Many 
snial  1  er-scal  e  computers  are  priced  attractively .  but  do  not  provide  the 
necessary  power. 

In  evaluating  candidate  architectures,  millions  of  operations  pet- 
second  (MOPS)  was  used  as  the  unit  of  measurement,  rather  than  millions 
of  Instructions  per  second  (MIPS),  because  the  MIPS  rating  can  be  mis¬ 
leading.  Machine  1  may  have  a  higher  MIPS  rate  than  machine  2,  but  it 
may  take  machine  1  more  Instructions  to  do  a  given  amount  of  work. 
Machine  2  could  actually  have  a  MIPS  rate  that  is  less  than  machine  1 
and  yet  accomplish  an  entity  of  work-  in  a  shorter  period  of  time. 

A  MOPS  figure  of  0.15  (150,000  operations  per  second)  and  an  1/0 
rate  of  24.0  (24  million  bits  per  second)  were  established  as  minimums 
needed  to  meet  the  functional  requirements  for  the  low-cost  system. 

These  figures  were  arrived  at  in  the  following  manner.  In  the  current 
AIS  system,  the  hardware  provides  functional  capabilities  that  are  much 
in  line  with  the  training  scenario  and  design  goals  described.  On  the 
current  AIS  there  are,  normally,  approximately  500  students,  approxi¬ 
mately  50  on-line  terminals  (including  CMl  terminals),  and  response 
times  similar  to  those  given  previously.  Therefore,  the  capabilities  of 
the  current  AIS  system  are  known.  The  CDC  6400  (equivalent  to  the  AIS 
Cyber  73)  which  was  selected  as  the  benchmark  system,  yielded  MOPS  * 

0.17  and  I/O  rate  =  24.0.  For  adequate  performance  in  the  low-cost 
system,  a  MOPS  figure  of  0.15  and  an  1/0  rate  of  24.0  were  determined 
to  be  minimums  below  which  unacceptable  degradation  of  performance  would 
resul t. 

After  evaluating  the  candidate  CPUs,  the  Digital  Equipment  Corp. 
(DEC)  VAX  11/780  was  selected  for  purposes  of  developing  a  system  design 
and  cost  analysis  for  the  low-cost  CAI/CM1  system.  Although  this  is  not 
the  only  CPU  that  will  meet  the  functional  requirements  for  the  low-cost 
system,  the  VAX  11/780  serves  as  the  example  CPU  in  subsequent  sections 
of  this  study. 

If  the  functional  capabilities  required  of  the  low-cost  system  were 
reduced,  some  corresponding  decrease  in  computing  power  (and  in  CPU 
cost)  could  be  realized,  for  example,  if  it  were  desired  to  support  the 
same  students  but  with  terminal  response  tines  of  10  seconds  instead  of 
the  faster  times  assumed  as  qoals  for  the  low-cost  system,  then  a  less 
capable  CPU  could  be  chosen.  It  should  be  kept  in  mind,  however,  th.it 
the  mainframe  represents  only  about  15''  of  the  cost  of  the  system.  Re¬ 
ducing  the  computational  capability  of  the  system  would  reduce  the  cost, 
but  the  responsiveness  of  the  system  would  also  be  reduced.  For  these 


reasons,  it  is  more  cost  effective  to  choose  a  system  which  will  yield 
the  higher  level  of  productivity  at  a  small  increase  in  cost.  The 
terminals  represent  about  50i  of  the  cost  of  the  system.  Major  cost 
reductions  can  only  be  achieved  by  eliminating  major  areas  of 
capability  (CAI  or  CMI,  for  example),  thereby  reducing  the  cost  for 
terminals. 

4.2  SURVEY  OF  INTERACTIVE  TERMINALS 

The  capabilities  required  from  interactive  terminals  by  students, 
authors,  instructors,  and  other  administrative  personnel  were  analyzed. 
Administrators  need  alphanumeric  (text)  presentations,  principally,  and 
have  very  little  need  for  any  additional  capabil ities.  Preparation  and 
presentation  of  CAI  lessons,  and  on-line  preparation  of  materials  for 
off-line  presentation,  also  rely  principally  on  alphanumeric  text 
capability.  However,  there  are  some-content  areas  and  special  presen¬ 
tations  which  will  benefit  from  color  and  graphics  capabilities.  In 
general,  CAI  programs  require  upper  and  lower  case  characters,  numbers, 
and  punctuation  from  the  American  Standard  Code  for  Information  Inter¬ 
change  (ASCII)  set,  and  a  -next-  Ley  for  student  control  over  the  pro¬ 
gram.  Programs  for  CMI  have  the  sane  requirements,  plus  needs  for 
additional  function  keys  (-back-  and  at  least  three  additional  function 
or  control  keys). 

The  attributes  of  some  300  alphanumeric  display  terminals  and 
graphics  terminals  were  reviewed  for  similarities  and  for  industry-wide 
standard  features.  The  results  (below)  indicate  constraints  within 
which  the  low-cost  system  will  have  to  be  designed  if  off-the-shelf 
terminals  are  to  be  used.  The  relative  merits  of  various  terminals  were 
evaluated,  and  candidate  terminals  are  recommended  for  use  by  students, 
authors,  and  administrators  in  the  low-cost  CAl/CMl  system. 

4.2.1  Display  format 

A  format  of  510  characters  per  line  is  standard  for  most 
terminals.  Alphanumeric  terminals  commonly  provide  24  or  25  lines,  and 
graphics  terminals  corononly  provide  34  or  40  lines.  Most  terminals 
provide  as  standard  either  64  different  displayable  characters  (numbers, 
upper  case  alphabet,  and  limited  punctuation),  or  the  standard  96  char¬ 
acter  AST;  l  set  (nvmbers,  upper  and  lower  case  alphabet,  and  extra 
symbols).  The  96  characters  are  considered  necessary  for  the  text 
requirements  of  the  low-cost  CAI /CMI  system. 

Symbols  are  formed  within  a  dot  matrix  on  all  the  terminals. 
Characters  using  hiqher  resolution  formats  are  generally  more  readable, 
especially  for  such  characters  as  ’’m"  and  "w."  Lower-case  letters  arc 
easier  to  read  when  the  descenders  for  the  characters  "q,"  "j,"  "p," 

"q,"  and  "y"  extend  below  the  matrix  used  with  capitals  and  numbers. 

The  low-cost  graphics  terminals  reviewed  have  a  picture  element 
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(pixel)  of  resolution  which  Is  some  fraction  of  a  one-character  matrix. 
The  range  of  resolution  standardly  available  ranges  from  the  lowest, 

160  pixels  horizontally  by  192  pixels  vertically,  up  to  the  highest 
resolution,  640  horizontally  by  476  vertically. 

Many  programs  used  on  the  low-cost  CAI/CMI  system  would  benefit 
from  some  accenting  method  for  portions  of  the  programs--! ines  to  serve 
as  separators  In  CMI  editors  or  to  box  in  accented  words  for  CAI  pro¬ 
grams,  for  example.  Most  of  the  low-cost  alphanumeric  terminals  did 
not  provide  line  drawing  capabilities,  but  some  provide  either  two 
brightness  levels,  or  the  capability  to  reverse  the  character  video 
for  individual  fields.  Words,  sentences,  or  phrases  can  be  put  into 
foreground  or  background  by  using  one  of  the  two  methods.  The  terminals 
considered  as  candidates  for  the  low-cost  system  have  at  least  one  of 
these  two  features  available. 

All  programs  arc  expected  to  send  an  X-Y  coordinate  to  position 

the  start  of  words  or  phrases,  or  to  echo  characters  back  to  the  user. 

Therefore,  candidate  terminals  should  have  addressable  cursors.  There 
is  no  standard  practice  for  setting  the  X-Y  position.  Various  sequences 
are  used  to  initialize  the  cursor  setting  mode,  and  different  values 
are  sent  to  the  terminals  in  either  X  first  or  Y  first  order  to  set  the 
coordinates . 

An  additional  requirement  is  that  both  alphanumeric  and  graphics 
terminals  must  have  the  ability  to  erase  the  screen  on  command  from  the 
computer  ami  from  the  terminal  user.  All  terminals  reviewed  nad  bulk 

screen  erase,  and  most  could  erase  to  the  end  of  line  or  the  end  of  the 

screen  from  the  cursor  position. 

4.2.2  Transmission  Speed 

A  transmission  speed  of  approximately  120  characters  per 
second  is  recommended  when  intermixing  upper  and  lower  case  alphabet 
and  numbers.  Transmission  of  ASCII  codes  over  communication  lines  re¬ 
quires  the  electronic  formatting  of  the  eight  bit  ASCII  code  between 
a  start  and  a  stop  hit.  To  transmit  120  characters  per  second,  at 
least  1200  bits  per  second  must  be  received  at  the  terminal.  All  ter¬ 
minals  considered  for  the  low-cost  system  were  able  to  receive  data 
in  a  switch-selectable  range  of  *10  to  9200  o>-  more  bits  per  second, 
ami  all  included  the  1200  bits  per  second  rate.  All  the  terminals 
included  electronics  to  receive  or  send  data  using  an  RS-232C  inter¬ 
face  to  a  modem  or  data  concentrator. 

-.2.3  Other  Transmission  Parameters 

To  give  the  computer  con.pl etc  control  over  the  characters 
returned  to  too  terminal,  and  to  provide  for  data  integrity  of  the 
user's  input  to  the  computer,  full  duplex  transmission  should  he  used. 

In  full  dai lex,  each  character  code  is  sent  to  the  computer  as  soon  as 


a  key  has  been  pressed.  The  computer  stores  the  single  key  value  and 
returns  for  display  either  the  same  code  or  some  other  code  which  is 
program  controlled.  For  example,  during  log  on,  a  computer  may  not 
return  the  password  for  display  but  may  instead  return  a  special  char¬ 
acter. 

Full  duplex  also  speeds  up  verification  of  data  received  by  the 
computer.  Half  duplex  transmission,  not  recommended  for  the  low-cost 
system,  customarily  places  the  keyboard  character  on  the  screen 
directly,  and  in  parallel  sends  the  character  code  to  the  computer. 

The  computer  does  not  return  a  character,  so  the  user  cannot  verify 
that  the  conputer  has  received  the  transmission  as  it  was  entered  at 
the  terminal. 

Full  duplex  character-ct-a-time  mode  also  implies  that  data  are 
sent  asynchronously ,  with  no  need  for  synchronization  with  the  network 
processing  unit. 

4.2.4  Keyboard 

All  terminals  reviewed  has  a  standard  typewriter  keyboard, 
with  letters  positioned  in  the  standard  QWLRTY  sequence.  A  special 
requirement  noted  for  the  low-cost  CAI/CKI  system  programs  is  that  a 
function  key  method  should  be  available  to  send  special  non-display 
character  commands  to  the  central  computer  for  CAI  or  CMI  program  or 
editor  control.  Most  of  the  low-cost  terminals  include  a  numeric  pad 
and  some  provide  a  number  of  function  keys.  Most  of  the  keyboards  pro¬ 
vide  a  control  key,  so  that  special  functions  can  be  simulated  by  non- 
displayable  control  characters  even  if  no  function  keys  are  available. 

4.2.5  Interactive  Terminal  Selection 

final  selection  of  interactive  alphanumeric  terminals  must 
be  based  on  specific  user  needs  and  applications.  The  described 
functions  are  considered  essential  in  any  selection  for  a  low-cost  CAI/ 
CMI  system.  To  configure  an  example  low-cost  system,  the  results  from 
the  foregoing  survey  were  used  in  selecting  candidate  alphanumeric  and 
graphics  terminals  which  meet  the  functional  requi rements .  However, 
these  are  not  the  only  terminals  which  could  satisfy  the  requirements. 
The  candidate  terminals  for  the  example  low-cost  system  are  as  follows: 

1.  Alphanumeric  text  terminal  for  students,  instructors,  and 
a Jmi ni strators--ADDS  Regent  103  terminal. 

2.  Text/limited  resolution  graphics/color  terminal  for  students-- 
Intecolor  8091G  terminal  with  lower  case  and  function  keys. 

3.  High  resolution  graphics  terminal  for  student  use--Toktronix 
4025  terminal  with  192  words  of  graphic  memory. 
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4.3  MANAGEMENT  DEVICES 


A  wide  variety  of  devices  can  serve  to  accomplish  the  required  CM  1 
functions.  In  fact,  if  keyboard  input  for  data  is  sufficient  and  If 
hardcopy  outpjt  is  not  needed,  the  student  terminals  can  accomplish  the 
CM1  functions.  The  configuration  of  management  devices  which  can  best 
serve  the  needs  of  a  low-cost  system  installation  will  depend  on  such 
factors  as  the  following: 

1.  Types  of  input  to  be  handled--manual  input  from  keyboard, 
pencil -marked  forms,  penci 1 -marked  cards,  badges,  off-line 
response  devices,  etc. 

2.  Typos  of  output  requi red- -hardcopy  output,  display  at  inter¬ 
active  terminal,  specialized  displays,  etc. 

3.  Quantity  of  input  and  output  to  be  handled--nuntber  of  students, 
frequency  of  interaction,  amount  of  information  handled  at  each 
interaction,  etc. 

4.  Instructional  and  administrative  decisions--proceduros  for 
registration,  absence  reporting,  testing,  prescribing  in¬ 
struction,  managing  student  progress,  etc. 

In  order  to  configure  an  example  system  without.  knowing  the  char¬ 
acteristics  of  the  installation  environment,  it  is  assumed  that 
capability  approximately  equal  to  too  current  AIS  management  devices 
will  be  required.  Each  of  the  management  devices  in  the  example  system 
will  therefore  consist  of  a  forms  reader,  a  line  printer,  and  the 
electronics  to  interface  with  tno  computer.  However,  the  management 
Jevices  needed  for  a  specific  installation  site  should  be  determined  by 
analyzing  the  needs  of  that  particular  site.  Considerable  cost  savings 
might  no  realized  if  simpler  management  devices  are  adequate  for  the 
Cfli  capability  required  at  a  site. 

4.4  COM 'UN  i  CAT  IONS  Kl  TWORK 

The  communications  network  is  considered  to  be  the  equipment 
necessary  to  connect  terminals  to  the  conrnmi cat  ions  processor,  lhis 
section  reviews  various  methods  of  transmitting  data  between  a  host 
computer  and  one  or  more  terminals  at  distances  up  to  h  miles,  the 
maximum  expected  between  buildings  for  the  low-cost  PA’/C'H  system.  It 
is  assumed  that  dedicated  lines  would  be  used  on  an  Air  lo»\e  base. 

Ig.iipnent  now  .  /ail able  provides  for  st  inlaid  d-Ma  fer  ial  and 
allows  for  transmiss  on  of  such  data  over  voice  grade  telephone  lines. 
The  ASCII  protocol  (line  discipline,  control  code  sequence  or  data 
♦ormat)  has  been  widely  accepted  by  government  and  industry  as  a  ven- 
•luii  ic  at  ions  standard.  A  system  using  modern  data  coin  unit  cat  ions 
formats  and  transmission  media  I  olds  the  iranuse  of  long  range  com- 
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pa tab i 1 i ty  and  expandability  between  CPUs  and  terminals. 

The  conmuni cat ions  system  for  the  low-cost  CAI/CMl  system  must 
provide  for  asynchronous  full  duplex  data  transmission  to  each  terminal 
at  120C  bits  per  second  (a  standard  speed  corresponding  to  120  char¬ 
acters  per  second)  to  allow  for  immediate  character  by  character  echo 
to  the  terminal  display.  Asynchronous  operation  describes  another 
factor  in  data  transmission--the  system  can  transfer  data  at  the  user's 
random  rate.  Full  duplex  describes  the  conmuni cations  system  as  having 
simultaneous  conruni cat ions  both  to  and  from  the  terminal,  similar  to  a 
telephone  conversation  between  two  parties. 

Modern  approaches  to  data  conmuni  cation  involve  the  use  of  readily 
available  telephone  circuits.  The  mcst  promising  options  available  are 
to  provide  one  or  more  dedicated  line  pairs  from  the  central  computer 
site  to  the  terminals.  Additional  line  amplifiers  and  correction  of 
chase  differences  are  not  required  since  the  overall  distance  between 
the  central  computer  and  any  terminal  is  expected  to  be  under  5  miles. 
Standard  telephone  pairs  as  presently  in  use  with  the  AIS  system  are 
expected  t.  •  be  available  for  the  low-cost  CAI/CMl  system.  Full  duplex 
operation  requires  a  pair  of  two-conductor  telephone  lines  to  provide 
a  continuous  communication  link  in  both  directions  between  computer  and 
terminals. 

The  tv.’O  sections  following  describe  line  modulation  techniques  for 
handling  data  between  the  central  computer  and  the  terminals.  The 
first  section  discusses  single  modems  for  coniruni eating  with  single 
stand-alone  terminals.  The  second  discusses  the  recently  developed 
statistical  multiplexer  systems  which  permit  operation  of  multiple  ter¬ 
minals  on  single  communication  line  pairs. 

4.4.1  Transmission  to  Single  Terminals 

Most  host  computer-related  hardware  can  transmit  data  to 
terminals  at  distances  up  to  about  200  feet.  However,  terminals 
located  at  greater  distances  should  not  be  operated  on  lines  with  raw 
digital  data,  because  of  signal  attenuation  and  susceptibility  to  noise. 
In  order  to  transmit  data  to  terminals  at  longer  distances,  modem  pairs 
are  recommended.  Modems  provide  for  conversion  of  the  digital  data 
by  frequency  shift  or  phase  shift  keying  of  an  audio  frequency. 
Applicable  types  cf  modems  include  trunk-line  and  short-haul  types. 

Bo tli  types  provide  the  capabilities  required  for  the  low-cost  C/I/CMl 
system,  including  full  duplex  asynchronous  modes,  diagnostics,  and 
interconnection  to  standard  OS- 232001  ectronic  interfaces. 

Short-haul  modems  generally  acconmodate  a  wide  range  of  trans¬ 
mission  speeds  and  do,  therefore,  contribute  to  a  system's  potential 
for  oxapnsion.  Speeds  typically  range  from  1200  to  19,200  bits  per 
second  and  are  switch  selectable.  The  price  range  for  short-haul 
modems  is  about  $6°0  to  $990  per  unit.  These  modems  can  be  used  in 


applications  requiring  high  bit  rates,  such  as  some  graphics  appli¬ 
cations,  and  are  good  for  distances  up  to  2C  miles  over  unconditioned 
lines.  Trunk-line  modems  are  generally  limited  in  their  range  of  trans¬ 
mission  speeds.  This  type  of  modem  sells  in  the  range  of  $320  to  $525 
each,  operates  at  or  below  2400  bits  per  second,  is  good  for  long 
distances  (up  to  several  thousand  miles),  but  requires  conditioned  lines 

The  following  modems  were  investigated  to  determine  their  capa¬ 
bilities  for  self-diagnosis  in  the  event  of  system  failure,  ease  and 
ability  in  matching  the  unit  to  the  telephone  lines,  and  ease  and  speed 
of  maintenance.  All  of  the  listed  modems  are  full  duplex,  have  asyn¬ 
chronous  operation  and  RS-232C  interfacing,  provide  diagnostics,  and 
have  a  minimum  speed  of  1200  bits  per  second. 


Trunk  Line  Modems 

Manufacturer 

Model  No. 

Speed  (Raud) 

Timeplex 

202 

Op  to  2000 

Tele- Dynamics 

7202? 

Up  to  1800 

Syntech 

TT-202 

Up  to  1 uoo 

Novation 

202 

1200/1000 

Penri 1 

1000 

Up  to  1000 

4.4.2  Transmission  to 

Multiple  Terminals 

Statistical  mu! 

tiplexors  (data  concentrators) 

are  ava'lab 

off-the-shelf.  The  units  handle  sampling  based  on  active  users  onlv, 
without  wasting  time  segments  on  idle  or  inactive  terminals.  The 
traffic  demand  is  monitored  by  a  statistical  algorithm  and  data  are 
moved  according  to  demand.  A  pair  of  inexpensive  statistical  multi¬ 
plexers  can  provide  service  over  2  telephone  circuits  to  as  many  as  32 
terminals. 

Traditional  error  correction  techniques  can  be  applied  with 
statistical  multiplexing.  In  the  event  that  an  error  occurs  in  the 
data  format  due  to  a  disturbance  in  the  communications  syten,  the 
statistical  multiplexer  will  retransmit  the  data.  The  multiplexers 
can  interrogate  each  other  to  repeat  a  message  to  be  sure  the  nessaoe 
is  transmitted  correctly. 

The  statistical  multiplexer  approach  utilises  a  pair  of  statistical 
multiplexers  (one  at  the  host  computer  and  the  other  near  the  terminals! 
two  modems,  and  a  pair  of  two-conductor  telephone  lines.  This  con¬ 
figuration  will  typically  handle  four 'or  more  terminals. 

for  the  statistical -multiplexer  approach,  the  modem  pair,  the 
communication  lines,  and  the  multiplexer  line  ports  are  required  to 
operate  at  only  one-hall"  the  bit  rate  of  the  maximum  composite  terminal 
speed.  When  one  to  eight  1200  bits  per  second  terminals  are  connected 


to  an  eight  channel  statistical  multiplexer,  for  example,  a  channel 
speed  of  only  4G00  bits  per  second  (3  x  120C/2)  is  required.  The 
eight  channel  statistical  multiplexer  system  will  operate  in  a  degraded 
mode  while  five  or  more  terminals  are  all  receiving  data  at  the  same 
instant.  Short-haul  modems  are  preferred  because  of  their  ability  to 
operate  at  faster  speeds  at  the  flip  of  a  switch. 

System  expansion  up  to  the  capacity  of  the  multiplexer  involves 
considerably  less  cost  than  adding  modem  pairs  as  in  the  single  terminal 
system.  While  it  may  be  necessary  to  purchase  add-on  channel  cards  to 
expand,  it  is  not  necessary  to  purchase  modems  and  lines  to  service 
another  terminal.  For  the  low-cost  system  with  relatively  short 
expected  distances  from  a  central  computer  to  terminals,  only  multi¬ 
plexer  systems  for  3  or  16  terminals  have  been  considered  so  that  the 
maximum  line  speed  will  not  exceed  9600  bits  per  second. 

The  statistical  multiplexers  and  short-haul  modems  listed  in  the 
following  tables  were  investigated  to  determine  their  self  diagnostic 
capabilities,  ease  and  speed  of  maintenance,  and  ability  to  easily  match 
characteristics  to  the  telephone  lines.  A  pair  of  multiplexers  and  a 
pair  of  modems  arc  required  for  a  system.  The  first  item  in  each  list 
was  selected  for  the  example  low-cost  system. 


Statistical  Multiplexers 


Manufacturer 

flodel  No. 

flax  No.  of 

Oigi tal 

Conmuni cat ions 

Terminals 

Associates 

Smart/Mux  115 

16 

Infotron 

Supemux  430 

8 

Mi  com 

S00 

8 

T ineplex 

M-8 

C 

Short  Haul  Modems 


Manufacturer 


Model  No. 


Syntech 

Tele-Dynamics 

Paradyne 

Codex 


L DM- 7296 
7300 
SRM-192 
8200  IDSU 


» 

1 
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5.0  SOFTWARE  SURVEYS  AND  ANALYSES 


5.1  SUPPORT  SOFTWARE 

Support  software  can  be  defined  as  all  of  the  software  supporting 
the  operation  of  the  low-cost  system  except  the  instructional  (CAI  and 
CMI)  software.  It  includes  the  operating  system,  file  manager,  loaders, 
linkers,  additional  compilers,  assembler,  and  utility  routines. 

Support  software  for  one  system  is  rarely  executable  on  another 
system  because  of  machine  dependencies.  Therefore,  support  software  is 
not  an  area  of  system  capability  which  should  be  evaluated  in  isolation 
from  the  CPU.  Rather,  the  capability  of  support  software  is  one 
criterion  to  be  considered  in  selecting  a  CPU.  If  possible,  the  CPU 
manufacturer's  support  software  should  be  used,  instead  of  creating 
special  purpose  software  or  modifying  the  manufacturer's  software.  This 
is  advantageous  for  the  following  reasons: 

1.  The  manufacturer  is  obligated  to  perform  maintenance  on  the 
support  software,  thereby  relieving  the  user  of  this  respon¬ 
sibility. 

2.  Manufacturers  often  make  improvements  to  their  support  software. 
As  reasonable  requests  from  the  user  community  are  received,  a 
manufacturer  often  implements  the  enhancements,  and  makes  the 
improvements  available  to  users.  If  the  support  software  is 
not  user-modified,  new  releases  of  software  from  the  manu¬ 
facturer  will  not  require  modification,  thereby  saving  effort 

in  acquiring  upgraded  support  software. 

3.  If  the  instructional  software  can  execute  without  being 
heavily  dependent  on  special  purpose  software,  advantage  can 
be  taken  of  advances  in  hardware  technology  and/or  lowered 
hardware  costs  by  readily  transferring  to  an  attractive  new 
system.  For  example,  if  the  execution  of  the  instructional 
software  requires  the  services  of  a  special  purpose  operating 
system,  then  the  instructional  software  is  heavily  dependent 
on  that  operating  system.  Moreover,  the  instructional  soft¬ 
ware  cannot  be  readily  transferred  to  a  new  system  if  the 
target  machine  does  not  support  the  language  in  which  the 
operating  system  is  coded.  On  the  other  hand,  if  the  execution 
environment  provides  standard  interfaces  to  manufacturer 
supplied  operating  systems,  then  the  instructional  software 

can  be  readily  transferred  to  a  new  system. 

The  support  software  for  the  example  CPO  (the  PEC  VAX  11/730)  was 
evaluated,  and  appears  to  have  all  of  the  capabilities  required  for  the 
low-cost  system.  It  is  therefore  used  as  the  support  software  for  the 
example  system  configuration. 


5.2  PROGRAMMING  LANGUAGE 


Evaluating  a  progranming  language  to  determine  its  merit  is  a  sub¬ 
jective  process.  Factors  entering  into  the  evaluation  of  a  language 
can  be  conflicting.  For  example,  a  language  with  many  available 
features  could  be  deemed  powerful  and  therefore  "good"--or,  because  of 
its  many  features,  the  same  language  could  be  viewed  as  overly  complex 
and  verbose.  The  main  difficulty  is  that  a  language  is  not  "good"  or 
"bad"  in  absolute  terms.  Previous  work  in  progranming  language  eval¬ 
uation  has  shown  that  the  merit  of  a  progranming  language  should  be 
determined  with  respect  to  (a)  the  application  area  for  which  it  will 
be  used  (in  this  case,  CMI  and  CAI  applications)  and  (b)  the  computing 
environment  in  which  the  language  will  appear  (the  hardware  equipment, 
the  fiscal  resources,  and  the  personnel  available). 

5.2.1  Top  Level  Language  Goals 

Definition  of  the  goals  for  a  progranming  language  should 
begin  by  considering  the  application  area  and  previous  research  into 
what  qualities  make  a  language  good.  Then,  by  examining  the  stages  of 
the  program  development  process  with  respect  to  the  application  area, 
and  identifying  the  most  important  objectives  for  a  language  at  the 
various  stages,  the  top  level  goals  can  be  defined. 

In  the  CAI/CMI  application  area,  the  stages  of  the  program  develop¬ 
ment  process  are  problem  specification,  design,  coding,  debugging,  and 
maintaining.  In  this  context,  the  following  top  level  goals  for  the 
low-cost  CAI/CMI  system  progranming  language  are  established: 

1.  Ease  of  Writing  CMI--Some  capabilities  are  particularly 
important  in  the  design  and  coding  stages  of  CMI  progranming. 
These  include  (a)  allowing  adequate  record  and  file  handling 
capabilities  for  student  data,  (b)  supplying  student  pre¬ 
scription  and  pacing  constructs,  and  (c)  providing  class¬ 
ification  and  typing  of  data  and  execution  paths. 

2.  Ease  of  Writing  CAI--Some  capabilities  are  particularly  im¬ 
portant  in  the  design  and  coding  stages  of  CAI  progranming. 
These  include  (a)  student  response  judging  and  analysis,  (b) 
easily  created  and  displayed  screen  frames,  and  (c)  easily 
created  questions  and  decision  branching. 

3.  F;el  iabi  1  i  ty--The  reliability  goal  is  applicable  during  coding, 
debugging,  and  maintaining.  A  languaqe  is  reliable  if  it 
facilitates  writing  correct  programs.  Compile  time  checking, 
execution  time  checking,  program  testing  features,  and  pro¬ 
gram  debugging  features  are  factors  contributing  to  this  goal. 

4.  Ease  of  Learning--Ease  of  learning  is  applicable  to  all  the 
stages  of  program  development.  It  encompasses  uniformity  of 
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syntax  and  context  consistency.  Unnecessary  complexity  of 
syntax  and  semantics  should  be  avoided.  A  fairly  common  set 
of  constructs  should  be  utilized,  supporting  as  few  linguistic 
concepts  as  possible.  For  example,  x  =  1  is  a  construct  that 
could  also  be  used  in  the  iterative  construct 

FOR  x  *  1  STEP  1  TO  10  00  (  ). 

The  language  should  not  contain  inconsistencies  or  context 
dependencies  (e.g.,  a  language  that  allows  int  =  1  where  "int" 
is  of  type  integer,  but  does  not  allow  stng  =  "ABCD"  where 
"st.ng"  is  of  type  string,  is  inconsistent  in  the  area  of 
assignments) . 

5.  Maintainability — The  maintainability  goal  is  applicable  in  the 
debugging  and  maintaining  stages.  A  language  that  supports 
maintainability  provides  readability,  ease  of  source  statement 
change,  and  easily  constructed  source  statements. 

6.  £fficiency--Ef ficiency  is  relevant  in  the  coding,  debugging, 
and  maintaining  stages.  The  efficiency  of  a  language  can  be 
measured  by  comparing  the  code  the  compiler  generates  (with 
respect  to  storage  and  speed)  to  that  produced  by  a  good 
assembly  language  programmer, 

5.2.2  Candidate  Languages 

Surveys  of  existing  programming  languages  (Sammet,  1978; 
Brahan,  1973)  were  used  to  determine  the  set  of  languages  which  might 
possibly  meet  the  top  level  language  goals.  Languages  which  could  not 
possibly  meet  the  goals  (e.g.,  those  designed  for  structural  engin¬ 
eering  or  linear  programming)  were  eliminated.  Also,  languages  which 
might  meet  the  goals  but  were  not  currently  implemented  on  at  least 
one  computer  were  eliminated.  Languages  which  have  never  been  imple¬ 
mented,  or  were  once  implemented  but  are  no  longer  supported,  were  con¬ 
sidered  to  be  too  costly  to  make  operational.  The  resulting  set  of 
candidate  languages  is  listed  below: 

ALGOL  63  -  A  powerful,  general  purpose,  extensible  language  used  to 
solve  a  large  number  of  programming  problems. 

BASIC  -  A  very  simple  language  used  primarily  in  solving  numerical 
problems,  but  with  some  advanced  features. 

CAMIL  -  A  general  purpose  interactive  language  used  for  CAl/CMl, 
systems,  and  application  programming. 

COBOL  -  An  English-like  language  used  primarily  for  business  data 
processing  problems. 
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C0RAL66  -  A  general  purpose  language  used  primarily  for  solving 
algebraic  problems. 

COURStIWR ITER  III  -  A  simple  language  used  for  preparing  CAI  courses. 

FORTRAN  -  A  language  designed  primarily  for  numerical  computation  but 
used  in  many  other  problem  areas. 

JOVIAL  -  A  language  designed  primarily  for  numerical  computations  and 
data  processing. 

NATAL  -  A  course-authoring  language  used  in  the  preparation  of  CAI 
material s . 

PASCAL  -  A  general  purpose  programming  language. 

PILOT  -  A  simple  language  for  preparing  CAI  materials. 

PLAIIIT  -  A  language  (embedded  within  a  system)  for  preparing  and  pre¬ 
senting  on-line  individualized  instruction. 

PL/ I  -  A  general  purpose  language  used  to  solve  a  large  number  of 
programning  problems. 

TAL  -  An  authoring  language  used  to  present  CAI  materials. 

TUTOR  -  A  language  used  for  preparing  computer-based  educational 

materials . 

S.2.3  Minimum  Language  Features 

Language  features  were  determined  by  examining  the  necessary 
CAi/CMI  functional  capabilities  from  Appendix  C  and  determining  a  set  of 
suggested  features  tor  accomplishing  those  capabilities.  Appendix  G 
lists  the  CAI  and  CM I  functions  and  shows  the  language  Features  to 
implement  each  function.  From  Appendix  G,  it  is  possible  to  determine 
(a)  which  language  features  could  be  eliminated  as  functional  capa¬ 
bilities  are  eliminated  or  (b)  which  functional  capabilities  might  have 
to  be  eliminated  as  certain  language  features  are  eliminated.  The  26 
language  features  are  as  follows. 

1.  Built-in  data  types  of: 

UITEGCR  -  e.g.,  1,  10,  lb 
REAL  -  e.g.,  l.b,  6.10 
LOGICAL  -  TRIJL ,  FALSE 
STRING  -  e.g.  ,  "ABCDC" 

ARRAY  -  e.g.,  ary(l)  =  aryl(x) 

RECORD  -  e.g.  ,  rec.x=3 
CLASS  -  e.g.,  class(cl ,c2,c3) 
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SET  -  e.g.,  x  6  setl 

TEXT  -  long  strings  of  textual  data  convenient  for  represen¬ 
ting  large  volumes  of  data  (up  to  a  screen  page). 

Support  of  variable  data  -  It  is  necessary  to  allow  data  to 
take  on  many  values  throughout  the  execution  of  a  program. 

Many  programs  representing  CAI/CMI  functions  need  this  capa¬ 
bility.  An  example  from  PASCAL  is: 

x :  =  1 0 ; 

X:=Y. 

Support  of  constant  data  -  It  is  necessary  that  some  data 
remain  constant  throughout  execution  of  a  program.  An  example 
from  CAMIL  is: 

DEFINE  INTEGER  x=b;  X  cannot  be  changed  within  this  program. 

Explicit  declaration  of  variables  and  constants  -  It  is  desir¬ 
able  that  variables  and  constants  can  be  declared  within  a  pro¬ 
gram  prior  to  use.  This  practice  is  commonly  accepted  as 
contributing  to  trie  creation  of  reliable  software.  An  exam, pie 
from  PL/ I  is: 

DECLARE  !  FIXED  INTEGER. 

Definition  of  procedures  (subroutines)  and  functions  (pro¬ 
cedures  that  return  a  value)  -  Capability  to  define  procedures 
and  functions  is  needed.  This  allows  the  CAI/CMI  programs  to 
be  modular  and  therefore  reliable  and  maintainable.  An  example 
from  PASCAL  is: 

PROCEDURE  CALC  ( INVAL: INTEGER); 

BEGIN 

procedure  body 
EfID. 

Specification  of  user-defined  data  types  -  It  is  convenient  to 
have  built-in,  expandable,  user-defined  data  types  for  use 
throughout  programs.  This  feature  allows  creation  of  different 
types  of  data  beyond  the  built-in  data  types.  An  example 
from  CAMIL  is: 

TYPE  PACKED  RECORD  (INTEGER  I;  NUMBER  N)  RECTYPE. 

Packing  data  within  records  and  arrays  -  It  should  be  possible 
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to  specify  packing  data  within  records  and  arrays  to  preserve 
space.  See  feature  6  for  an  example  of  a  packed  record. 

G.  Automatic  allocation  and  access  for  Indexed  Sequential  files  - 
It  should  be  possible  to  specify  a  record  or  group  of  records 
to  begin  reading  or  writing  by  supplying  a  key  for  record 
positioning.  This  feature  is  necessary  for  efficient  manip¬ 
ulation  of  student  data  on  disk. 

9.  Automatic  allocation  and  access  for  Direct  Access  files  -  It 
should  be  possible  to  position  to  a  particular  reconi  by 
specifying  its  location  on  disk.  This  feature  is  necessary 
for  efficient  maintenance  of  student  data  on  disk. 

10.  File  sharing  -  It  shouldbe  possible  for  many  processes,  simul¬ 
taneously  active  on  the  CPU,  to  access  the  same  disk  data  file. 
This  feature  is  necessary  for  efficient  manipulation  of  student 
data  on  disk. 

11.  Record  reservation  -  It  should  be  possible  for  a  process  to 
lock  out  all  other  processes  while  updating  a  student  record 
on  disk.  This  feature  is  necessary  for  reliable  maintenance 
of  student  data  on  disk. 

12.  IF  THEN  ELSE  construct  -  Conditional  branching  (choosing  alter¬ 
nate  program  paths)  should  be  possible.  An  example  from  PASCAL 
is: 

IF  X  <  > Y  THEN  Z: =A  ELSE  Z:=B. 

13.  Iterative  statement  construct  -  Loops  should  be  possible  with¬ 
in  programs  using  an  iterative  construct.  Many  of  the  CAI/CMI 
functions  will  be  implemented  with  program  loops.  An  example 
from  FORTRAN  is: 

no  io  k-i  jo 

loop  body 
10  CONTINUE. 

14.  N-way  conditional  branching  -  It  is  necessary  to  be  able  to 
choose  one  alternate  out  of^many  to  implement  many  of  the 
functions.  An  example  from  CAMIL  is: 

CAST  x  or 
BEGIN 
1  y  «-  10; 

:>  ?  1 1,  • 
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5  a  5; 
END. 


15.  Assignment  statement  -  It  must  be  possible  to  assign  values 
to  variables  within  the  language.  This  feature  is  necessary 
for  virtually  all  functions.  An  example  from  JOVIAL  is: 

A1  a  A1  +  1$. 

16.  Program  interrupt  detection  -  A  construct  is  needed  to  allow 
for  user  interruption  of  the  normal  program  flow.  This  feature 
is  necessary  for  programs  implementing  CAI  functions.  An 
example  from  CAM1L  is: 

ON  FI  00 
BEGIN 

handle  the  function  key  interrupt 
END. 

17.  GOTO  statement  -  An  unconditional  branching  statement  is  needed 
in  the  language.  This  is  necessary  to  transfer  to  specified 
sections  of  code.  An  example  from  BASIC  is: 

GO  TO  5. 

IB.  Communication  with  CMI  terminals  -  The  language  must  be  capable 
of  directing  output  to  and  receiving  input  from  CMI  terminals. 
Many  CMI  functions  require  this  feature. 

19.  Communication  with  CAI  terminals  -  The  language  must  be  capable 
of  directing  output  to  and  receiving  input  from  CAI  terminals. 
Many  CAI  functions  require  this  feature. 

20.  Arithmetic  operations  -  The  functions  of  addition,  subtraction, 
multiplication,  division,  exponentiation,  absolute  value, 
modulus,  random  number  generation,  sine,  cosine,  tangent,  co¬ 
tangent,  square  root,  string  concatenation,  and  integer  shift¬ 
ing  should  be  available  for  use  in  CA]  ana  CMI  functions.  An 
example  from  PASCAL  is: 

X:=  Y/2. 

21.  Relational  operations  -  The  operations  of  equal,  not  equal,  less 
than,  greater  than,  less  than  or  equal,  greater  than  or  equal, 
and  set  membership  must  be  supported.  CAI  and  (Ml  functions 
use  these  operations.  An  example  from  CAM1L  is: 

IF  x  <  y  THEN  z  10. 
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Logical  operations  -  The  operations  of  not,  ami,  or,  exclusive 
or,  set  complement,  set  intersection,  and  set  union  must  he 
supported,  for  use  in  fAI  and  CHI  functions.  An  c<amp1e  from 
ALG0L63  is: 

If  X  <  10a  Y  >  5  THEN  7  <-  25. 

23.  Interactive  terminal  page  oriented  textual  coninunication  -  It 
must  be  possible  to  display  textual  information  by  the  page 
at  an  interactive  terminal  via  the  language.  CAI  functions 
require  that  text  be  displayed  at  student  terminals. 

24.  Terminal  graphics  -  A  basic  line  drawing  capability  which  can 
be  utilized  to  construct  terminal  graphics  should  be  resident 
within  the  language.  CAI  functions  may  require  this  feature. 

25.  Hardcopy  support  -  It  must  be  possible  to  interface  with  a 
hardcopy  printer  for  generating  reports,  program  listings,  and 
other  documents.  CAI  and  CMI  functions  require  this  feature. 

26.  Magnetic  tape  capability  -  It  is  necessary  to  maintain  laroe 
volumes  of  student  data  on  an  inexpensive  backup  medium,  by 
using  magnetic  tape,  areas  of  more  expensive  high-speed  disk 
storage  can  he  freed  of  student  data. 

5.2.4  Nontechnical  Criteria 

While  the  technical  merit  of  a  language  is  of  primary 
importance,  certain  non-technical  criteria  should  also  he  considered  in 
determining  the  cosi  of  implementing  the  C A l /CM I  functions  in  that 
language.  These  criteria  are  as  follows: 

1.  Language  availability  on  recommended  hardware  -  it  a  candidate 
language  is  not  implemented  on  the  host  computer,  there  is  a 
cost  associated  with  making  the  language's  compiler  operational 
on  the  system. 

2.  Quality  of  candidate  language  documentation  -  This  will  have  a 
bearing  on  how  quickly  the  software  can  tie  developed. 

3.  Availability  of  trained  progranmers  -  If  there  are  no  trained 
progranmers  available  for  a  candidate  language,  the  cost  of 
implementing  the  software  in  this  language  will  he  higher. 

4.  Availability  of  programs  providing  required  CAI /CMI  functional 
capabilities  in  a  candidate  language  -  If  software  to  implement 
CA I /CMI  functional  capabilities  already  exists  in  a  candidate 
language,  then,  with  respect  to  this  criterion,  the  cost  of 
using  that  language  will  be  decreased. 
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5.2,5  Language  Evaluations 

To  assist  in  evaluating  candidate  languages  for  the  example 
low-cost  system,  an  approach  developed  by  Brosgol ,  Hartman,  Nestor, 

Roth,  and  Weissman  (1977)  can  be  utilized.  This  is  an  excellent 
approach  for  forcing  evaluators  to  think  as  objectively  as  possible 
about  the  subjective  process  of  evaluating  a  programming  language.  In 
this  approach,  a  ratine  F'+rix  R,  an  application  vector  A,  and  an  inter¬ 
mediate  vector  I  are  ust  in  conjunction  with  a  language  vector  for  each 
candidate  language  to  derive  a  technical  merit  score.  The  technical 
merit  score  and  four  management  evaluation  scores  for  a  language  are 
entered  into  a  language  score  vector  which,  in  conjunction  with  a 
language. management  vector,  yields  a  final  score  for  each  candidate 
language.  The  following  paragraphs  describe  this  approach,  and  suggest 
suitable  parameters  for  use  in  evaluations  for  the  low-cost  system. 

Matrix  R  is  a  26  by  6  matrix  whose  rows  represent  the  language 
features  (section  5.2.3)  and  -  se  columns  represent  the  top  level 
language  goals  (section  5.2  Each  matrix  element,  Rij,  is  a 
numeric  value  indicating  the  extent  to  which  language  feature  i  con¬ 
tributes  toward  accomplishing  language  goal  j.  The  application  vector 
is  a  column  vector  of  6  entries  representing  the  relative  contributions 
of  the  top  level  goals  toward  the  CAI/CMI  application  area.  After  deter¬ 
mining  R  and  A,  then  R  x  A  produces  a  column  vector  I  with  26  entries, 
each  entry  representing  the  contribution  of  a  language  feature  toward  the 
application  area. 

Each  entry  in  the  rating  matrix  R  is  a  value  between  0  and  10, 
indicating  how  much  language  feature  i  contributes  toward  accomplishing 
language  goal  j.  The  range  0  to  10  is  chosen  to  allow  the  final  tech¬ 
nical  merit  values  to  fall  in  the  range  of  0  to  150  (the  maximum 
technical  merit  score  is  actually  149.54).  This  range  could  be  greater 
or  less  and  still  be  representative  of  the  technical  merit  of  a  lan¬ 
guage.  However,  0  to  150  is  a  reasonable  range  to  work  with,  since 
technical  merit  values  are  then  neither  unduly  large  nor  extremely 
small.  The  suggested  rating  matrix  R,  for  evaluating  languages  for  the 
low-cost  system,  is  as  follows: 
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The  application  vector  A  is  a  column  vector  of  six  entries,  each 
entry  value  reflecting  the  contribution  of  a  top  level  language  goal 
toward  the  CAI/CM1  application  area.  The  vector  entries  must  sum  to 
1.0,  and  reflect  the  judged  Importance  of  a  goal  with  respect  to  the 
application  area.  Entries  should  avoid  placing  too  much  importance 
on  potentially  overlapping  goals  (e.g.,  reliability,  learnability, 
maintainability).  The  recommended  entries  for  application  vector  A  are 


i 


.18 

.18 

.18 

.14 

.16 

.16 
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Since  the  entries  in  R  represent  the  contributions  of  the  language 
features  toward  each  of  the  top  level  language  goals,  and  since  the 
entries  in  A  represent  the  contributions  of  each  top  level  language 
goal  toward  the  CAI/CMI  application  area,  then  k  x  A  produces  a  column 
vector  I  with  entries  representing  the  contributions  of  the  language 
features  toward  the  application  area.  The  product  of  the  recommended 
vectors,  R  x  A,  is 


6.74 

7.96 

4.12 
5.16 

7.38 

5.38 
5.66 
5.76 
4.04 
5.06 
5.40 
7.72 
7.88 
7.86 
8.04 
4.44 
1 .46 
6.00 

5.96 
6.58 

7.12 
6.04 
5.98 
3.32 
4.48 
4.00 


A  language  vector  CL i  is  then  produced  for  each  candidate  language. 
The  entries  in  the  language  vector  indicate  the  extent  to  which  the 
language  features  are  provided.  The  product  CL i  x  1  will  then  produce 
a  score  inoicating  the  technical  merit  of  the  language  with  respect  to 
the  CAI/CMI  application  area.  These  scores,  ranking  the  technical 
merits  of  candidate  languages  with  resoect  to  the  CAI/CMI  application 
area,  can  then  be  used  in  management  evaluations  to  arrive  at  a  can¬ 
didate  language  recommendation. 

The  j th  entry  of  language  vector  CLi  indicates  how  candidate  lan¬ 
guage  i  contributes  toward  the  Jth  language  feature.  Cor  each  language 
feature,  the  "degree  of  compliance"  for  a  candidate  "language  must  be 
estimated.  A  suggested  approach  for  making  these  estimates  is  as 


follows : 

T  :  The  language  totally  meets  the  requirement.  A  score  of  1.0 
will  be  recorded  in  the  language  vector. 

P  :  The  language  partially  meets  the  requirement.  A  score  of  from 
0.1  to  6.9  will  be  recorded  in  the  language  vector  according 
to  the  degree  the  requirement  is  partially  fulfilled. 

F  :  The  language  fails  to  meet  the  requirement.  A  score  of  0.0 
will  be  recorded  in  the  language  vector. 

U  :  It  is  unknown  from  the  available  documentation  whether  the 
requirement  is  satisfied  or  not.  A  score  of  0.0  will  be 
recorded  in  the  language  vector  since  poor  documentation 
should  be  considered  even  if  the  feature  is  implemented  but 
not  documented. 

After  t'Li  is  established,  the  technical  merit,  TMi ,  of  a  candidate 
language  with  respect  to  the  CA I /CM I  application  area  can  be  calculated 

TMi  -  Cl.  i  x  I . 

Development  costs  for  implementing  the  CAI/Cf'I  softwar  must  bo 
considered.  Costs  can  be  categorized  as  follows: 

1.  Acquiring  the  implementation  language. 

2.  Training  the  programmers  in  the  given  language. 

3.  Designing  the  system. 

<4.  Coding  the  system  in  the  given  language. 

5.  Testing,  debugging,  and  verifying  the  system. 

6.  Maintaining  and  upgrading  the  system. 

The  entries  in  a  Language  Management  vector,  LM,  reflect  the  con¬ 
tributions  of  technical  merit  and  of  the  nontechnical  criteria  to  lower 
ing  system  development  costs.  The  recommended  LM  vector  is 


Technical  merit  :  .33 
Language  availability  on  recommended  hardware  :  .25 
Quality  of  documentation  :  .05 
Availability  of  trained  programmers  :  .06 
Availability  of  instructional  software  in  the  language  :  .31 
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A  five-entry  Language  Score  row  vector,  lS,  whose  entries  are 
scores  ranging  from  0  to  100,  is  established  for  each  candidate  lan¬ 
guage.  Each  entry  in  LS  describes  how  well  a  candidate  language  is 
judged  to  contribute  toward  the  entries  of  LM.  The  first  entry  in  LS 
is  the  technical  merit  score  for  the  language,  translated  to  a  0 
to  100  scale.  The  other  entries  in  LS  are  determined  by  examining  a) 
the  languages  implemented  on  the  candidate  hardware  system  (for  purposes 
of  this  example  system,  the  VAX  11/780),  b)  the  available  language 
documents,  c)  the  availability  of  trained  programmers,  and  d)  the  re¬ 
quired  CAI/CMI  functional  capabilities  for  possible  existing  implemen¬ 
tation  by  candidate  languages.  This  product  LS  x  LM  is  the  final  score 
for  a  candidate  language. 

Clearly,  selecting  a  language  for  CAI/CMI  application  involves  a 
considerable  amount  of  subjectivity.  However,  if  the  foregoing  pro¬ 
cedures  are  followed,  the  evaluation  process  is  made  visible,  the 
numbers  entered  into  the  process  are  open  to  inspection,  and  the 
results  Cdn  be  accepted  or  rejected  on  their  merits.  Also,  the  eval¬ 
uation  process  is  designed  to  allow  change  to  the  vector  values  (the 
process  could  easily  be  automated).  In  this  way,  opinions  of  different 
individuals  can  be  considered  before  a  final  language  recommendation  is 
made. 

In  selecting  a  language  for  the  low-cost  system,  the  technical  and 
nontechnical  merits  of  the  candidate  languages  should  be  considered  in 
the  context  of  the  defined  application  area.  The  evaluation  matrix 
approach  just  described,  or  another  explicit  technique,  should  be  used 
to  reflect  the  weightings  assigned  to  the  technical  and  nontechnical 
merits  of  the  languages  under  consideration.  The  actual  selection  must 
consider  specific  user  applications,  program  costs,  minimum  language 
features,  etc.  Any  language  that  meets  these  criteria  may  be  selected. 

As  part  of  configuring  an  example  low-cost  system,  the  technical 
merits  of  the  candidate  languages  were  calculated,  relying  in  some  cases 
on  first-hand  experience  with  a  language  and  in  other  cases  on  published 
user  literature.  The  nontechnical  criteria  discussed  previously  were 
also  estimated,  ending  with  calculations  of  final  scores  for  the  can¬ 
didate  languages.  After  application  of  the  matrix  approach  and  the 
parameters  described  above,  CAMIL  was  selected  for  the  example  system 
conf igurat.oa. 
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5.3  APPLICATIONS  SOFTWARE 

5.3.1  Applications  Programs  Required 

To  implement  the  CAI/CMI  functions,  the  following  types  of 
application  software  will  be  required: 

1.  CM  I  software  -- 

0  Interface  with  the  interactive  terminals. 

0  Interface  with  the  management  terminals. 

0  Validation  of  forms  that  are  submitted. 

0  Scoring  of  tests  (on-line  and  off-line). 

0  Resource  management. 

0  Assignment  selection. 

0  Management  of  students  to  time  targets. 

2.  CAI  software  -- 

0  CAI  presentation  program. 

0  CAI  authoring  system. 

0  Graphics  editor. 

3.  Editor  capability  to  display  and  modify  -- 
0  Course  definition  files. 

0  Student  data  files. 

0  Test  keys. 

0  Resource  files. 

4.  Report  generation  programs  -- 

0  CAI  lesson  evaluation  report. 

0  Training  management  reports. 

0  Course  Evaluation  Sunmary. 
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Test  Item  Evaluation  report. 

0  Learning  Center  rosters. 

0  Student  block  progress  reports. 

0  Absence  report. 

0  Homework  summary. 

0  A  general  data  retrieval  and  report  program. 

5.  '.lata  base  support  software  -- 

0  An  automatic  CMI  data  base  validation  program. 

0  A  program  to  remove  student  performance  data  from  disk  and 
generate  the  tape  and  disk  file  for  the  generalized  data 
retrieval  program  and  for  the  Course  Evaluation  Summary. 

0  A  program  to  remove  the  CA1  data  from  the  disk  and  generate 
the  tape  files  for  the  CAI  report  programs. 

0  A  program  to  remove  test  item  data  from  the  disk  and  generate 
the  tapes  used  by  the  Test  Item  Evaluation  reoort  program. 

®  An  archiver  to  remove  CAI  lessons  that  are  not  in  regular 
use  from  the  on-line  data  base  and  record  them  on  tape. 

0  A  program  to  bring  archived  CAI  programs  back  as  they  are 
needed. 

0  An  archiver  to  take  programs  that  are  not  in  regular  use 
from  disk  and  record  them  on  tape. 

0  A  program  to  bring  archived  programs  back  as  they  are  needed. 

5.3.2  Selection  of  Applications  Programs 

A  number  of  existing  CBI  systems  were  surveyed  to  determine 
their  capabilities  in  the  applications  programs  area.  All  of  the 
existing  systems  have  some  elements  of  capability  that  can  be  charactei- 
ized  as  CMI,  and  most  have  some  CAI  capability.  All  op  the  functional 
capabilities  required  for  the  low-cost  system  have  bc-en  implemented  on 
one  or  more  of  the  existing  systems,  but  no  single  existing  system  in¬ 
corporates  all  of  the  capabilities. 

A  large  proportion  of  the  CAI  and  CMI  functional  capabilities  for 
the  low-cost  system  are  currently  implemented  in  the  set  ot  applications 
programs  for  the  A1S.  It  is  possible,  with  the  addition  of  applications 
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pi  ogramiiing,  to  bring  the  A I S ,  or  another  CBI  system,  up  to  the  level 
required  for  the  low-cost  system.  The  survey  of  existing  capabilities 
indicated  that  fewer  programs  would  have  to  be  added  to  the  A  IS  soft¬ 
ware  than  to  other  existing  systems.  Therefore,  the  AIS  applications 
software  is  selected  for  the  example  system  configuration. 

6.0  SYSTEM  ANALYSIS 

Integrating  the  functional  requirements  with  the  hardware/software 
capabilities  was  approached  by  deter  lining  the  system  requirements 
dictated  by  the  full  list  of  functions.  Then,  the  additional  system 
requirements  dictated  by  the  candidate  support  software  and  programming 
language  were  assessed.  Finally,  cho  total  system  requirements  were 
compared  with  cha  capabilities  of  the  lano'idate  hjruware  to  determine 
che  feasibility  of  utilizing  that  hardware  with  the  recommended  soft¬ 
ware  and  language  to  accomplish  the  required  f.A I  and  CM1  functions. 

The  cAI/CKI  functions  and  their  payoff  ratings  were  used  as  a 
starting  joint  in  determining  the  system  requirements .  An  attempt  was 
made  to  determine  the  costs,  in  terms  of  storage,  I/O  time,  CPU  time, 
or  actual  collars,  of  each  function.  However,  the  functions  are  so 
heavily  interrelated  that,  in  most  cases,  it  is  net  possible  to  assign 
a  cost  to  a  single  function.  Fur  example,  the  information  in  o 
single  file  may  service  several  functions.  If  any  one  of  those 
functions  is  to  be  included,  that  file  must  be  included  also,  but  the 
file  can  then  be  snared  by  many  other  functions  without  any  additional 
cost.  In  other  words,  costs  are  driven  by  groups  of  functions,  a.id  in 
general,  a  cost  will  be  incurred  if  any  or  all  of  a  group  of  functions 
are  to  be  provided.  Eliminating  one  function  from  among  a  group  does 
not  usually  result  in  decreased  system  cost.  Therefore,  in  subsequent 
analyses,  groups  of  functions  are  the  usual  unit  of  analysis  rather 
than  single  functions.  However,  if  a  single  function  represents  a 
considerable  impact  on  the  system,  and  can  be  isolated,  then  it  is 
treated  in  that  way.  A  list  of  the  functions,  with  the  files  required 
and  approximate  CPU  times  needed  for  each,  is  shown  in  Appendix  C. 

These  estimates  derive  from  measured  times  and  known  file  requirements 
in  the  current  AIS.  The  detailed  requirements  are  sunned  and  used  as 
a  basis  for  sizing  the  system  hardware. 

6.1  DISK  SPACE 

Required  disk  space  can  be  estimated  from  a  knowledge  of  the  files 
required,  the  required  record  structures,  and  the  number  of  records 
needed.  Experience  on  the  AIS  and  the  structures  of  the  AIS  courses 
were  used  as  models  in  estimating  storage  space  requirements.  Para¬ 
meters  used  were  number  of  courses,  number  of  students  per  course,  and 
numbers  of  blocks  and  lessons  per  course.  Information  on  the  config¬ 
uration  of  specific  target  courses  was  not  available.  Therefore,  some 
assumptions  consistent  with  possible  application  sites  in  Air  Force 
resident  technical  training  were  made: 
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1.  Five  courses  implemented  on  the  system. 

One  hundred  students  per  course  per  shift  (500  students  on 
CMI/CA1  at  any  one  time). 

3.  Five  blocks  of  instruction  per  course. 

4.  Sixty  lessons  per  block. 

Some  subparameters  within  these  main  headings  were  also  assumed, 
based  on  current  practice: 

1.  Two  shifts  of  operation. 

2.  Students  are  assigned  into  learning  centers,  with  25  students 
in  each  center. 

3.  Three  course  versions  (shredouts,  for  example)  per  course  are 
implemented. 

Five  categories  were  used  in  estimating  disk  space  requirements: 

1.  Active  programs. 

2.  CAI . 

3.  CMI. 

4.  Recent  Data  File  (RDF),  the  on-line  record  of  the  most  recent 
student  completions  of  tne  blocks  and  courses. 

5.  Support  for  the  compiler,  operating  system,  file  management 
system,  and  the  necessary  support  software. 

The  number  of  active  programs,  assuming  a  lower  software  mainten¬ 
ance  loac  than  on  tne  current  AIS,  was  estimated  to  bo  approximately 
80X  of  current  AIr'  usage,  for  a  total  of  30  million  bytes  (3  Dits  per 
byte) . 

Disk  requirements  for  tne  CAI  programs  were  estimated  using  the 
current.  AIS  structure  as  a  jasis,  but  with  a  7 ST  safety  margin.  This 
total  then  is  30  mill. on  ..ytes. 

The  CMI  requirements  were  also  estimated  from  the  AIS  basis,  but 
with  a  safety  margin  of  50T  (CMI  requirements  con  be  predicted  with 
greater  accuracy).  The  CMI  requirement  is  30  million  bytes. 

Tne  Recent  Data  File  requirement  depends  upon  how  many  student 
records  the  user  activity  viill  maintain  on-line.  An  estimated  22 
million  bytes  will  be  required  in  order  to  maintain,  on-line,  the 
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records  of  the  most  recent  £00  students  to  complete  each  of  the  courses 
and  each  of  the  blocks  assumed  for  the  low-cost  system. 

Support  for  the  compiler,  the  operating  system,  ihe  file  management 
system,  and  the  necessary  support  software  will  require  67  million  bytes 
of  disk  storage.  T hi-  includes  a  50"  allowance  for  expansion,  should  it 
be  desirable  to  expand  the  system  software  capabilities  to  provide  "nice 
to  have"  features  in  addition  to  the  necessary  features. 

In  suimiary,  the  total  requirement  for  disk  space  for  the  low-cost 
system,  assuming  i:J0  students  it.  the  RPC,  is  179  million  bytes  of 
storage. 

6.2  TERMINALS 

The  assumptions  teat  there  will  be  TOO  students  on  the  system  at 
any  one  time,  and  that  10'  of  the  instruction  will  he  CAI,  lead  to  a 
requirement  for  SO  student  interactive  terminals.  A  reasonable  mix  for 
these  is  assumed  to  be  40  alphanumeric  terminals,  7  low  resolution 
graphics/color  terminals,  and  3  high  resolution  graphics  terminals. 

Nine  additional  alphanumeric  terminals  are  assumed  to  he  required  for 
admini strati ve  use. 

The  CM I  functions  can  bo  accomplished  with  a  mix  of  on-line  and 
off-line  management  terminal  devices,  but  the  requirements  will  depend 
on  specific  user  applications.  To  input  off-line  tests  and  provide 
printed  feeaback/prescription  after  each  such  input,  a  low-cost  printer 
using  inexpensive  teletype  paper  is  reconnended ,  in  conjunction  with  a 
forms  reader,  CRT,  or  other  peripheral s.  Five  such  management  terminal 
sites  will  be  required  to  manage  600  students  per  shift. 

In  summary,  the  following  64  terminals  would  ho  incorporated  in  a 
"typical"  configuration  of  the  low-cost  system  for  resident  technical 
-raining: 

1.  Forty  interactive  a  1 phanumer a c  terminals  for  student  use. 

2.  Seven  low  resolution  graphics/color  terminals  for  student  use. 

3.  Three  high  resolution  graphics  terminals  for  student  use. 

4.  Nine  interactive  alphanumeric  terminals  for  administrative 
use . 

5.  Ti ve  management  terminal  sites. 

6.3  MEMORY  REQU!REMrNTS 

Memory  requirements  are  driven  by  such  aspects  of  the  system  as 
characteristics  of  the  operating  system,  number  of  terminals,  computer 
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architecture,  and  language.  The  CAI/CMI  functions  are  significant 
factors  because  (a)  they  determine  the  requirements  for  terminals,  (b) 
the  types  of  presentations  necessitated  by  the  functions  affect  the 
amount  of  memory  necessary,  and  (c)  there  will  be  many  processes, 
representing  the  applicat  on  programs  which  implement  the  functions, 
operating  concurrently  on  the  system.  In  line  with  the  design  goals 
and  assumptions,  there  will  be  64  users  on  the  system  at  maximum  load 
(users  at  50  CAI  terminals,  5  management  terminal  sites,  and  9  admin¬ 
istrative  terminals).  Therefore,  there  will  be  a  maximum  of  64 
application  processes  on  the  system  concurrently.  A1S  experience  has 
shown  that  processes  similar  to  those  representing  the  application  pro¬ 
grams  implementing  the  recommended  CAI/CMI  functions  will  run  with 
adequate  but  not  extravagant  efficiency  (i.e.,  producing  the  desired 
response  times)  when  29,000  to  30,000  bytes  of  central  memory  are 
allocated  to  each  process.  Therefore,  the  central  memory  requirement 
due  to  the  CAI/CMI  functions  is  1.9  million  bytes. 

Support  for  the  compiler,  the  operating  system,  the  file  manage¬ 
ment  system,  and  the  necessary  support  software  requires  100,000  bytes 
of  central  memory.  Much  of  the  operating  system  will  continuously  be 
in  central  memory,  but  the  compiler,  file  management,  and  support  soft¬ 
ware  will  be  loaded  only  as  required. 

The  total  central  memory  requirement  due  to  the  CAl/Cill  functions 
and  to  software  support  is  2  million  bytes. 

6.4  DATA  RATES 

The  recor, mended  functions  (Appendix  C)  were  reviewed,  within  the 
framework  of  A I S  experience,  as  to  the  signal  input  and  output  require¬ 
ments  for  the  various  types  of  transactions,  and  the  approximate  page 
lengths  to  be  displayed.  Integrating  the  results  indicates  that  a  data 
rate  of  1200  bits  per  second  will  be  sufficient  for  most  terminal  dis¬ 
plays.  Complicated  graphics  displays,  depe  ding  on  the  instructional 
requirements  and  on  the  terminal  capabilities,  could  require  a  higher 
rate.  Therefore,  an  option  for  higher  speeds  is  a  factor  for  consider¬ 
ation  in  the  design  of  the  system. 


6.5  TAPE  DRIVE  REQUIREMENTS 


The  number  of  tape  drives  required  is  determined  by  two  types  of 
usage:  main  shift  and  off-shift  usage.  Main  shift  usage  will  consist 
of  requests  for  data  submitted  by  course  personnel,  requests  to  restore 
archived  lessons,  and  requests  to  restore  archived  programs.  The  re¬ 
quests  for  data  will  include  course  evaluation  summaries,  test  item 
evaluation  reports,  CAI  reports,  and  special  retrieval  requests.  Most 
of  these  data  will  be  on  tape.  It  is  possible  to  meet  this  requirement 
with  one  tape  drive,  but.  a  tape  drive  is  required  for  each  process  that 
is  running.  Two  drives  will  allow  two  requests  to  be  run  at  the  same 
time. 


The  off-shift  requirements  include  batch  jobs  submitted  by  the  on- 
shift  personnel  and  periodic  jobs  to  maintain  the  CA1,  CMl,  and  proqram 
data  bases.  The  periodic  jobs  will  include  the  following: 

1.  Remove  the  CMl  sunmary  data  from  disk  and  merge  wi th  the 
history  tapes. 

2.  Remove  the  CAI  summary  data  from  disk  and  merge  with  the 
history  tapes. 

3.  Remove  the  test  item  data  from  disk  and  merge  with  the  history 
tapes, 

4.  Archive  CAI  lessons  that  have  not  teen  used  within  a  designated 
retention  period. 

b.  Archive  computer  programs  that  have  not  been  used  within  a 
designated  retention  period. 

Although  requirements  could  be  met  with  one  tape  drive,  there  would 
be  severe  penalties.  These  include: 

1.  Only  one  report  from  tape  could  be  run  at  a  time. 

2.  A  scratch  area  on  a  disk  will  be  required  to  store  data  during 
merges . 

3.  Computer  operators  will  have  substantially  more  loading  and 
unloading  of  tapes  from  drives. 

4.  If  one  drive  breaks,  no  tape  work  can  be  done. 

With  two  drives,  two  reports  can  be  processed  at  the  same  time. 

The  CPU  is  normally  faster  than  one  tape  drive  ano  can  process  data  from 
one  drive  while  reading  data  from  the  other  drive.  The  system  would 
also  be  able  to  go  tape  to  memory  to  tape,  with  two  tape  drives.  With 
one  tape  drive,  the  process  is  tape  to  memory  to  disk  to  memory  t.o  tape- 
whlch  is  substantially  slower.  Therefore,  two  tape  drives  are 
recommended  for  the  low-cost  CA I /CMl  system. 

6.6  HARDCOPY  PRINTOUT  REQUIREMFNTS 

The  reports  required  for  the  five  courses  assumed  to  be  Implemented 
on  the  low-cost  system  can  be  handled  easily,  on  an  off-shift  basis,  by 
a  medium  speed  line  printer,  At  600  lines  per  minute,  production  will 
be  approximately  600  pages  of  printout  per  hour.  This  is  more  than 
adequate  to  support  Information  retrieval  and  analysis  requests.  There¬ 
fore,  a  line  'rinter  capable  of  about  600  lines  per  minute  is 
recormended . 


7.0  CONFIGURATION  OF  AN  EXAMPLE  SYSTEM 


The  preceding  sections  establish  the  hardware,  software,  and 
functional  requirements  for  the  low-cost  system,  and  designate  candidate 
hardware  and  software  items  for  a  low-cost  system.  The  following  para¬ 
graphs  configure  an  example  system  capable  of  meeting  the  requirements 
and  design  goals  as  previously  described. 

7.1  MAINFRAME  AND  PERIPHERALS 

The  example  CPU,  the  DEC  VAX  11/780,  is  capable  of  adequately  but 
not  extravagantly  fulfilling  the  requirements  for  the  low-cost  CAI/CM1 
system.  The  example  system  will  include  mainframe  and  peripherals  as 
fol lows : 


Cost  Monthly 

Maintenance 


One  standard 

VAX  11/780  package 

wi  th: 

1.  51 2K  bytes  of  memory. 

2.  One  RCP06  176  million 
byte  disk  drive  with 
MASSBUS  adapter. 

3.  One  TEF16  45  inch/second 
tape  transport  with 
MASSBUS  adapter. 

4.  One  D7.11-A  asynchronous 
multiplexer  for  8  lines. 

5.  One  LA36  PLCwriter  II 


console. 

Additional  equipment  to 
meet  hardware  requi remcnts : 

1.  One  MS780DB  51 2K 

$185,000 

$827 

byte  memory  expansion. 

2.  One  MS 70000  1  ,024K 

22,000 

no 

byte  memory  expansion. 

3.  One  RCM03-AA  07  million 
byte  disk  unit 

35,000 

:20 

with  MASSBUS  adapter. 

4.  One  TE16-AE  4S  inch/ 

25,000 

I  70 

second  tape  transport  . 

5.  One  1  I’ll -DA  90  character 

11  ,,’90 

t'O 

660  1  ine/mimite  printer-, 
it .  One  071 1 -B  asynchronous 

25.700 

185 

mul ti plexor  for  8  1 inos . 

1 ,800 

21 

1 


Cost 


Monthly 

Maintenance 


7.  Three  DZ11-E  asynchronous 

multiplexers  for  16  lines 
at  $3,850. 

$  11,550 

$  75 

0.  One  H9602-DF  UNI  BUS 

options  cabinet. 

2,300 

none 

Totals 

$319,640 

$1 ,668 

7.2  TERMINALS 

The  "typical"  system  configuration  (section  6.1.2)  includes  9 
administrative  terminals,  40  alphanumeric  text  terminals,  7  low 
resolution  graphics/color  terminals,  3  high  resolution  graphics 
terminals,  and  5  management  terminal  sites.  If  the  administrative 
terminals  are  the  same  as  the  student  text  terminals  (a  feature  that  is 
recommended  to  facilitate  substitutions  in  case  of  failure),  terminal 
requirements  become  (a)  49  alphanumeric  terminals,  (b)  7  limited 
graphics/color  terminals,  (c)  3  high  resolution  graohics  terminals, 
and  (d)  5  management  terminal  sites. 

T rom  section  4.2.5,  the  example  alphanumeric  text  terminal  is  toe 
ADDS  Regent  100  with  function  key  option.  The  example  low  resolution 
qraphics/color  terminal  is  the  Intecolor  00016  with  lower  case  and 
function  keys.  The  example  high  resolution  graphics  terminal  is  the 
Tektronix  4025  with  0192  words  of  graphics  memory.  The  costs  for  stu¬ 
dent  and  administrative  terminals  for  the  "typical"  configuration  are: 

49  ADDS  Regent  100  at  SHOO  =  $53,900 
7  Intecolor  8001G  at  S2700  =  $10,900 
3  Tektronix  4025  at  $4495  =  $13,485 


$86,285 

For  purnoses  of  configuring  an  example  system,  it  is  assumed  that 
each  of  the  five  management  terminal  sites  will  consist  of  an  0RSCAN  17 
optical  forms  reader,  a  Teletype  Model  40  R0P  printer,  a  Dioital  Equip¬ 
ment  Corporation  LSI-11  mini-computer  with  peripheral  cards,  and  various 
displays,  push  buttons,  and  specially  designed  circuitry.  Each  such 
terminal  will  oust  approximately  $26,000.  Total  cost  tor  the  five 
management  terminal  sites  will  be: 

5  Special  Management  Terminals  at  $26,000  =  $130,000. 
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7.3  COMMUNICATION  SYSTEM 

A  64-port  system  for  59  student  and  administrative  terminals  and 
five  management  terminal  sites  is  required  for  the  example  system.  It 
is  assumed  that  four  buildings  (work  areas)  will  be  used  in  the  oper¬ 
ational  low-cost  system,  with  the  computer  and  some  classroom  space 
in  one  of  the  buildings.  To  provide  connuni cat  ions  to  the  three  remote 
bulldinqs,  three  statistical  multiplexer  system*,  each  handling  16 
terminals,  will  be  used.  The  computer  building  is  assumed  to  have  16 
terminals  connected  directly  to  the  network  processing  equipment  of  the 
computer. 

The  configuration  to  each  of  the  remote  buildings  will  include  a 
pair  of  16-channel  statistical  multiplexers  (one  at  the  host  computer 
and  one  in  the  remote  building  connected  to  the  terminals),  a  pair  of 
short-haul  modems  (one  at  each  multiplexer)  operating  at  9600  bits  per 
second,  and  two  twisted  pair  telephone  lines.  The  entire  conmuni cat  ions 
configuration  will  then  consist  of  six  statistical  multiplexers,  six 
short-haul  modems,  and  six  pairs  of  Government  furnished,  twisted  pair, 
dedicated  telephone  lines.  Using  the  Digital  Communications  Associates 
(DCA)  Smart/Mux  115  statistical  multiplexers  and  the  Snytech  LDM-7296 
short-haul  modems  (section  4.4.2),  the  cost  of  the  example  communication 
system  is: 

6  OCA  Smart/Mux  115  at  $4300  =  $25,800 

6  Syntech  LDM-7296  at  $  690  =  $  4,140 


$29,940 


7.4  LIFE  CYCLE  COSTS 

The  initial  cost  of  the  hardware  represents  only  part  of  the  total 
cost.  Some  changes  in  the  facilities,  organization,  and  courseware  will 
add  to  the  initial  cost.  During  the  lifetime  of  the  system,  costs  will 
be  incurred  for  maintenance  and  supnort,  materials  and  supplies,  energy 
used,  and  initial,  as  well  as  some  continuing,  training  for  personnel 
using  the  system. 

Eight  years  is  used  as  the  total  life  of  the  system.  Costs  for 
hardware  and  software  maintenance  and  support  are  estimated  and 
escalated  for  those  H  years.  Some  costs  could  not  be  established 
because  of  lack  of  required  information.  For  example,  courseware  costs 
cannot  be  established  unless  detailed  information  about  the  target 
courses  is  known.  Also,  costs  per  work  hour  cannot  he  established  with¬ 
out  details  of  the  internal  costs  cf  the  organization  furnishinq  the 
manpower.  In  these  cases,  then,  a  work  hour  estimate  without  dollar 
costs  was  established. 


The  following  paragraphs  indicate  how  the  calculations  were  carried 


out,  and  show  estimates  in  work  hours  for  the  identified  tasks.  Where 
appropriate,  quantity  discounts  (over  10  units)  are  included.  However, 
savings  from  original  equipment  manufacturer  (OLM)  agreements,  which 
might  be  effected  on  all  equipment,  are  not  included. 

7.4.1  Mainframe  and  Peripheral  Hardware 

Initial  costs  of  the  computer  mainframe  and  peripherals  (not 
including  terminals  and  conmunication  equipment)  are  listed  helow. 
Maintenance  costs  for  successive  years  arc  calculated  by  increasing  each 
previous  year's  cost  by  o, ,  the  percentage  of  increase  projected  by 
Digital  Equipment  Corporation  for  salary  and  spare  parts  increases. 


Initial  Cost  (Section  7.1  above)  $319,640 
Year  1  maintenance  cost  CO, 010 
Year  2  maintenance  cost  21,216 
Year  3  maintenance  cost  22,489 
Year  4  maintenance  cost  23 ,839 
Year  5  maintenance  cost  2b  ,269 
Year  b  maintenance  cost  26 , 788 
Yea*'  7  maintenance  cost  28,392 
Year  8  maintenance  cost  30,09!) 


Total  8  year  cost  $61  7,741 

7.4.2  Software  Development 

The  two  main  areas  of  software  development  for  the  low-cost 
system  are  the  transportability  of  the  AIS  developed  software  and  the 
additional  applications  programs  necessary  to  moot  the  functional 
requirements  of  the  low-cost  system.  The  transportabi 1 i ty  of  the  Als 
CAMIL  software  is  presently  being  carried  out  by  the  McDonnell  Douglas 
Astronautics  Company.  The  additional  applications  software  is  estimated 
to  represent  three  work  years  of  effort.  This  figure  includes  all 
functions  reconmended  for  the  low-cost  system  (Appendix  (')  and  not  pro¬ 
vided  by  current  AIS  applications  software,  except  interfacing  with 
other  Air  force  management  systems. 

7.4.3  Software  Maintenance 

For  purposes  of  estimating  the  cost  of  maintaining  the  soft¬ 
ware  of  the  example  system,  one  full  time  application  prograwuer  and  one 
full  time  system  orograimier  are  indud<  l.  The  maintenance  will  include 
minor  Improvements  to  the  software  as  well  as  corrections  to  faulty 
software  (due  to  programning  errors  in  the  initial  software). 

Should  expansion  of  the  recoinuendcd  system  occur  at  one  Air  force 
base,  the  two  software  maintenance  personnel  described  in  t fie  preceding 
paragraph  could  maintain  four  systems,  because  the  additional  systems 


are  duplicates  of  the  initial  system,  changes  or  corrections  made  to 
one  can  easily  be  applied  to  the  remaining  systems. 

If  multiple  versions  of  the  initial  system  should  be  implemented  at 
various  bases,  additional  software  maintenance  personnel  would  be  re¬ 
quired.  Two  maintenance  personnel  would  be  necessary  at  the  initial 
base,  as  described  above,  with  one  additional  application  programmer  at 
each  additional  base  to  implement  the  modifications  determined  by  the 
personnel  at  the  initial  base. 

7.4.4  Operations 

When  hundreds  of  students  and  instructors  enter  data  into  a 
computer,  "foul-ups"  are  inevitable.  This,  plus  software  and  hardware 
failures,  establishes  a  need  for  monitoring.  A1S  experience  confirms 
the  need  for  monitors  to  be  located  in  the  classrooms  near  the  terminal 
areas  and  in  the  computer  room. 

One  person  with  a  thorough  knowledge  of  the  operation  of  the 
computer  system,  stationed  near  the  terminals,  can  greatly  improve  the 
operation  of  the  system  and  save  on  instructor  hours.  This  person 
should  be  available  at  the  terminal  for  correcting  minor  student/' 
computer  interface  problems.  This  frees  the  instructor  to  handle  the 
student's  instructional  problems.  This  person  also  is  responsible  for 
keeping  the  terminals  in  working  order,  and  for  calling  maintenance 
with  any  of  the  more  difficult  operational  problems.  The  number  of 
persons  required  for  this  task  depends  on  the  locations  of  the 
terminals.  If  several  management  terminals  are  located  near  each  other, 
one  person  can  handle  problems  at  several  terminals.  As  physical 
separation  grows,  additional  people  are  needed.  This  report  assumes 
that  the  user  group  will  furnish  these  personnel  and  that  they  will  be 
trained  as  needed  (section  7.4.10). 

Computer  operators  are  needed  at  central  site  to  lo.>  *  tapes, 
answer  maintenance  calls,  keep  records,  and  generally  monitor  computer 
operations.  Since  this  system  is  a  production  system,  it  is  expected 
that  operator  intervention  will  be  minimal,  limiting  batch  jobs  and 
other  tasks  that  require  operator  intervention  to  specified  times  of 
day  further  minimizes  the  manpower  requirement.  Tnc  effort  per  system 
can  be  minimized  if  the  mainframes  for  several  low-cost  systems  are 
located  near  each  other,  for  cost  estimation  purposes,  one  operator  per 
system  per  shift  and  one  additional  backup  operator  are  included. 

7,4."  Terminal  and  Communication  equipment 

The  terminals  and  convm.u on t ion >  equipment  require!  for  a 
"typical"  low-cost  CA1/C.M1  system  configuration  are  as  follows  (snare 
p/.rts  requi cements  are  listed  in  section  7.4.0): 
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4-3  ADDS  Reqent  100  at  $  1100  =  $  53,900 

7  Intecolor  G001G  at  <  2700  =  S  13,900 

3  "ektronix  4025  at  $  4495  =  $  13,485 

b  OCA  Smart/Mux  115  at  S  4300  =  $  25,800 

6  Syntech  LDM-7296  at  $  690  =  $  <1,140 

5  Management  Terminals  at  $26000  =  $130,000 


Total  Purchase  Price  $246,225 

7.4.6  Terminal  and  Communication  Equipment  Maintenance 

because  of  the  prol i feration  of  equipment  from  different 
vendors,  it  is  not  practical  to  procure  maintenance  contracts  from  each 
individual  vendor.  Most  manufacturers  require  that  malfunctioning 
equipment  be  delivered  to  their  own  service  centers.  The  manufacturer- 
service  approach  generally  means 

1.  High  cost  per  unit  maintenance  contracts  (usually  3/49  to  1\ 
of  purchase  price  oer  month). 

2.  Shipping  or  travel  lime  delays. 

j.  A  requirement  for  many  complete  unit  spares. 

The  recommended  maintenance  approacn  is  to  provide  an  on-site 
repair  capability  for  non-mainframe  equipment.  The  primary  duty  of 
these  personnel  is  to  maintain  individual  CRT  terminals,  communication 
equipment,  and  management  terminal  peripheral  equipment  (such  as  the 
forms  reader,  controller,  and  printer)  by  troubleshooting  the  equipment 
am  replacing  malfunctioning  printed  circuit  boards  or  individual  com¬ 
ponents.  Secondary  duties  include  accepting  trouble  calls  from  class¬ 
rooms,  and  locating  and  replacing  malfunctioning  equipment,  for  cost 
estimation  purposes,  4  work  years  per  year  are  included  for  these  tasks. 

Using  the  on-site  maintenance  personnel  approach,  a  supnly  of 
spare  parts  is  needed  for  CRT  terminals,  comuni  cation  equipment,  and 
mananement  terminal  component  equi ament  such  as  mark  readers,  con¬ 
trollers,  and  printers.  To  cover  spare  parts  for  all  equipment  except 
the  printers,  a  spare  parts  budget  of  T  of  purchase  price  per  year  is 
called  for.  A1S  experience  indicates  that  the  heavily  used  mechanical 
printers  require  a  261  per  year  budget,  since  the  entire  printer  must 
be  replaced  approximately  every  4  years. 

Using  a  rate  of  7%  of  purchase  price  per  year  for  a  spare  parts 
budget  for  non-mainframe  equipment  except  management  terminal  printers, 
which  are  figured  at  a  25“  annual  budget,  the  first  year  costs  arc  as 
fol lows : 


49  ADDS  Regent  100  at  $  77  =  $  3,773 

7  Intecolor  8001G  at  $189=$  1,323 

3  Tektronix  4025  at  $  315  =  $  945 

6  OCA  Smart/Mux  115  at  $  301  =  $  1,806 

6  Syntech  LDM-7296  at  $  48  =  $  288 

5  Management  Terminals-- 
Non-printer  equipment  at  $1540  =  $  7,700 

Printers  (257  rate)  at  $1000  -  $  5,000 


First  Year  Spare  Parts  Cost 

of  Non-Ma inframe  Peripherals  $20,835 

Using  an  appreciation  rate  of  7.57  per  year  for  the  increase  in 
cost  of  replacement  parts,  the  spare  parts  budget  for  the  first  8  years 
would  be  as  follows: 

For  year  1,  the  spare  parts  cost  is  $20,835 

For  year  2,  the  spare  parts  cost  is  $22,398 

For  year  3,  the  spare  parts  cost  is  $24,078 

For  year  4,  the  spare  parts  cost  is  $26,67u 

For  year  j,  the  spare  parts  cost  is  $23,677 

Tor  year  6,  the  spare  parts  cost  is  $30,828 

For  year  7,  the  spare  parts  cost  is  $33,140 

For  year  8,  the  spare  parts  cost  is  $35,625 


Total  spare  parts  appreciated  cost  is  $222,257 

7.4.7  Facility  Requirements 

Installing  a  computerized  training  system  in  a  resident 
technical  training  environment  sometimes  necessitates  facility  changes. 
Tne  changes  that  are  necessary  depend,  of  course,  on  the  existing 
physical  plant,  computer  location,  amount  of  mediated  material ,  and  many 
other  factors.  Without  known  parameters,  it  can  only  be  stated  here 
that  the  ideal  situation  is  large  learning  centers  located  close  to  the 
management  terminals  and  as  close  as  possible  to  the  computer  room. 

The  size  of  computer  room  needed  for  this  installation  is  approximately 
350  sguare  feet.  The  computer  room  must  be  air  conditioned  to  maintain 
approximately  70  degrees  F  with  a  48,303  BTU  per  hour  load. 

7.4.8  Energy  Requi remen ts 

The  energy  requirements  for  the  system  cons i s t  of  the  direct 
electrical  consumption  for  mainframe,  peripherals,  and  terminals,  and 
the  air  conditioning  for  the  computer  room.  Tne  total  direct,  electrical 
power  requirement  is  29  KWH  per  operating  hour.  The  air  conditioning 
requirement  is  for  43,000  BTU  per  hour. 
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7.4.9  Courseware  Development 

Introducing  a  CBI  system  into  an  on- oo i ny  course  of  instruc¬ 
tion  will,  almost  always,  entail  costs  for  changing  courseware.  These 
changes  will  vary  greatly  depending  on  tne  existing  environment.  For 
example,  an  extremely  important  factor  is  whether  a  course  is  currently 
self-paced.  Without  definition  of  target  courses,  these  costs  nave  not 
been  estimated. 

7.4. ID  Training 

The  success  of  any  system  is  largely  dependent  upon  the 
sophistication  of  its  users.  To  this  end,  training  costs  should  be  in¬ 
cluded  in  the  life  cycle  costs,  experience  with  the  operation  of  the 
A I S  has  identified  several  categories  of  training  that  are  hiqhly 
reconiiiended.  The  levels  of  training  end  the  estimated  hours  of  iiistruc- 
ticn  are  as  follows: 

1.  The  Deputy  Coi.nander  for  Training  (TTC/TT)  and  subordinate 
divisions,  e.g.,  the  Plans  and  Requi rements  Division  (TTCX), 
constitute  tne  upper  level  of  local  training  administration 
am  management:.  Training  at  this  level  should  consist  of  a 
system  overview  and  will  require  approximately  ft  instructional 
hours.  Technical  Training  Group  Commanders  and  personnel  of 
their  Resource  Management  Sections  (TTCC)  shoula  also  receive 
training  at  least  equal  to  that  provided  TTC/TT. 

2.  Branch  Chiefs  and  course  supervisors  are  the  next  level  of 
training  management  and  should  receive  a  more  complete  pre¬ 
sentation  of  the  aspects  of  the  system.  This  level  will  require 
approximately  SO  instructional  hours. 

3.  Curricula  and  Instruction  and  Measurement  personnel  should 
receive  training  in  the  evaluation  aspects  of  the  system.  This 
training  will  require  approx imately  120  instructional  hours. 

4.  Instructor  supervisors  and  instructors  snould  be  trained  in 
the  operational  aspects  of  the  system.  The  instructor  super¬ 
visors  should  receive  approximately  9  instructional  hours,  and 
the  instructors  should  receive  approximately  CO  hours  of 
instruction. 

b.  Course  Data  Case  Manager  and  Student  control  personnel  should 
receive  instruction  on  manipulat  ir.g  the  course  data  base  to  fit 
course  organization  and  neeos.  The  data  manager  will  be 
expected  to  make  all  data  base  changes  and  should  receive  120 
hours  or  training.  The  student  control  person  will  work 
directly  in  the  day-to-day  operations  with  tne  students  and 
should  receive  Id  hours  of  training. 
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6.  School  Data  Base  Management  personnel  should  receive  more 
global  training,  becoming  familiar  with  all  courses  and  pro¬ 
viding  leadership  and  services  in  such  areas  as  strategies, 
student  management,  documentation  training,  and  course 
liaison.  This  training  should  be  approximately  400  hours . 

7.4.11  Supplies 

Tne  necessary  consumable  supplies,  used  mainly  w:th  tPt 
management  terminals  and  off-line  testing,  or  :  (a)  central  site  printer 

paper,  (b)  management  terminal  printer  oaper,  (cl  student  test  ftir.-rs, 
and  (d)  miscellaneous  supplies,  such  as  printer  ribbons,  computer  tapes, 
dish  oacks,  tools,  and  cleaning  supplies.  Kstimates  of  these  costs  <ar£ 
based  upon  cunent  experience  with  AIS  in  usage  per  on-line  student- 
The  number  of  students  on-line  is  assumed  to  be  500  per  shift  with  two 
shifts.  The  estimated  costs  of  these  supplies  are  based  on  the  follow1"11!! 
assumptions: 

1.  Test  form  usage  is  estimated  based  on  the  assumption  of  five 
transactions/student/shift.  Total  forms  usage  is  then  calcu¬ 
lated  by  assuming  250  training  days  per  year,  with  302  waste 
and  failure.  Total  yearly  cost:  519,500. 

2.  f la na cement  terminal  printer  paper  costs  are  estimated  by  u5hwT 
the  five  transactions/shi ft  figure.  The  yearly  cost:  $6,b'.'0, 

J.  Central  site  printer  paper  usage  is  estimated  by  assuming  that 
a  production  system  will  not  have  software  development.  Linage 
will  include  CMI  reports,  CA I  authoring  and  evaluation, 
materials  development,  and  other  administrative  and  evaluation 
usage.  Usage  is  estimated  at  20  cf  current  AIS  usaqe.  TOUl 
yearly  cost:  $6,000. 

4.  Magnetic  tapes  and  dish  paexs  will  be  needed  at  tne  central 

site  for  data  storage  ana  backup.  It  is  estimated  that  Sf, JO  pet 
year  will  be  required  for  tapes.  Four  extra  disk  packs  should 
be  purchased  the  first  year  and  an  average  of  cue  per  year 
thereafter,  ineso  costs  are  detailed  below. 

1st  year  =  $2,7e0 

2nd  through  8th  jeer  -  $675  per  year 

6.  Miscellaneous  supplies  include  needs  for  day-to-day  opera ti^.s 
such  as  tools,  bench  stock,  cleaning  materials,  printer  ribbon;- 
etc.  The  estimated  yearly  miscellaneous  Supply  east:  $1  ,*jL;!0, 

The  summary  supply  costs,  escalated  at  7.5  per  year,  is  shpW,-, 
be  lew. 
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1st  year  $36,200 
2nd  year  $38,915 
3rd  year  $41 ,833 
4th  year  $44,970 
5th  year  $43,342 
6th  year  $51 ,967 
7th  year  $55,864 
8th  year  $60,053 
TOTAL  for  3  years  -  $378,144. 


7.4.12  Cost  Sunmary 


Total 

Recurri nq 

8  Year 

Costs , 

Paragraph/T i tie 

Cost  For 

Per  Year 

One  System 

7.4.1 

Mainframe  Peripheral  Hardware 

$517,741 

7.4.2 

Software  Development 

3  MY 

7.4.3 

Software  Maintenance 

16  MY* 

2  MY* 

7.4.4 

Operations 

16  MY* 

2  MY* 

7.4.5 

Terminal  i  Communication 
Equipment 

$246,225 

7.4.6 

Terminal  &  Communication 
Maintenance 

$222,257 

$27,782 

32  MY 

4  MY 

7.4.7 

Facility  Requirements 
for  Computer  Room 

350  sq.  ft. 

7.4.8 

Energy  Requirements, 

928  MWH 

116  MWH 

two  shift  operation 

1536  MET U 

132  MET U 

7.4.9 

Course  Development 

Unknown 

7.4.10 

Training 

Unknown** 

7.4.11 

Suppl ies 

$378,144 

$47,263 

SYSTEM  LIFE  CYCLE  COSTS: 

Hardware  &  Suppl ies 

$1 ,364,367 

$75,050 

Manpower 

67  MY 

8  MY 

*  Additional  systems,  if  co-located  with  a  first  system,  could  share 
personnel . 


**  Required  hours  will  depend  on  personnel  turnover  rates. 

8.0  CONCLUSIONS 

The  principal  components  of  a  dedicated  local  system  will  include 
°  Computer  and  peripherals. 

0  Student  and  administrative  terminals--al phanumerics ,  graphics/ 
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0  color,  and  high  resolution  graphics. 

o  devices— forms  reader,  printer,  and  controller. 

0  Comnunlcatlons  network. 

Software. 

An  operationally  configured  dedicated  local  system,  to  suDoort  5nn 

r  sh,ft  CHI  and  CAI  In  resident  tecbJlcal  train?"!  can  be 
acquired  for  approximately  $500,000  (1979).  a 
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APPENDIX  A 

LISTING  OF  CAI/CMI  FUNCTIONS  FOR  SURVEY 

I.  STUDENT  TESTING  ANO  EVALUATION  -  Using  the  computer  to  assist  the 
instructor  in  evaluating  the  student.  The  tests  which  are  read  and 
scored  by  the  computer  can  include,  for  example,  aptitude  tests,  lesson 
or  block  tests,  performance  checks,  attitude  measures,  or  biographical 
information.  Questions  could  be  true-false,  multiple  choice,  or  con¬ 
structed  response  (single  words,  or  phrases,  or  sentences).  Students 
receive  quick  feedback,  instructors  don't  have  to  score  tests  or 
record  grades,  and  all  test  results  are  readily  available  for  reports 
or  evaluations. 

A.  Test  Scoring  and  Feedback  Capabilities  -  The  computer  system 
can  score  tnose  tests  for  which  test  Leys  have  been  previously 
input.  The  results  from  each  test  can  be  printed  out  for  the 
student  and  instructor.  Questions  are  true-false  or  multiple 
choice. 

RATING: 


1.  Answers  are  scored  right  or  wrong,  and  a  designated  percent 

correct  is  required  to  pass  the  test.  RATING;  _ 

2.  Total  score  on  the  test  is  corrected  for  guessing. 

RATING:  _ 

3.  Some  questions  can  be  weighted  more  heavily  than  others. 

RATING: 

4.  A  designated  percentage  of  a  set  of  objectives  must  be 

passed  in  order  to  pass  the  test,  for  example  3  out  of  b 
objective.  RATING:  _ 

b.  A  specific  objective  or  objectives  must  be  passed  in  order 
to  pass  the  test.  RATING: 


t>.  A  specific  question  must  be  passed  In  order  to  pass  the 

objective  (or  the  test).  RATING:  _ __ 

7.  Performance  checklists,  as  completed  by  the  instructor,  can 
be  scored  by  the  computer.  RATING:  _____ 

B.  An  instructor  can  bypass  a  test  by  "certifying"  that  the 
student  knows  the  material.  RATING:  _ 

G.  Audi tjonaj  On-Line  Testing  Capabilities  -  Students  can  be  tested 
on'-Tine  at  a  computer  terminal.  The  order  of  questions  and  the 
sequence  of  alternate  answers  for  each  question  can  be  scrambled 
automatical ly  for  each  test  administration.  On  retesting  after 
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an  initial  failure,  retest  can  be  on  only  the  failed  objectives. 
Questions  can  be  true-false,  multiple  choice,  or  constructed 
response  (English  words).  The  amount  of  testing  can  be  depen¬ 


dent  on  the  student's  earlier  performance  in  the 
this  test. 

course  or  on 
RATING: 

1 . 

Random  presentation  order  for  questions. 

RATING: 

0 

U  • 

Random  presentation  order  of  alternative 
answers. 

RATING: 

3. 

Retesting  is  only  over  failed  objectives. 

RATING: 

4. 

Questions  can  require  constructed  response 
(English)  answers. 

RATING: 

5. 

< 

Amount  of  testing  depends  on  performance  on 
earlier  questions  in  this  test. 

RATING:  _ 

6. 

Amount  of  testing  depends  on  performance 
earlier  in  the  course. 

RATING: 

c. 

Determination  of  Pre-Course  Student  Characteristics  -  Provides 

for 

use 

of  student  data  collected  before  or  at  the  beginning  of 

training.  Uses  can  include,  for  example,  examining  eliminations 
as  a  function  of  aptitude  and  years  of  school  or  prediction  of  how 
fast  a  student  will  complete  the  course.  RATING: 


1.  Computer  storage  and  retrieval  of  relevant  Air  Force 
personnel  data  -  for  example,  ASVAB  or  AFQT  scores, 
or  biographical  data  such  as  previous  schooling. 

RATING: 

2.  Storage  and  Retrieval  of  Student's  Prior  Knowledge 

of  Course,  Block,  or  Lesson  Objectives  -  scores  on 
criterion-referenced  pretests  or  other  measures  of 
training  objectives.  RATING: 


3.  Storage  and  Retrieval  of  Student's  Learning  Process  Skills  - 
for  example,  any  ability,  attitude  or  interest  type  measures 
such  as  reading  comprehension,  study  habits,  etc. 

RATING:  _ 

4.  Storage  and  Retrieval  of  Student's  Course-Specific  Critical 
Entry  Skills  -  for  example,  aptitudes,  abilities  and  inter¬ 
ests  that  are  specifically  related  to  success  in  a  par¬ 
ticular  training  specialty,  such  as  mathematical  ability 

or  interest  in  the  career  field.  RATING: 


D.  Determination  of  Wlthin-Course  Student  Attitudes  and  Interests  - 
Provides  for  use  of  attitude  or  interest  data  collected  in  the 
course  of  training.  Uses  could  include  evaluation  of  instruc¬ 
tional  materials  or  procedures,  prediction  of  completion  times 
or  rates  of  progress,  and  prediction  of  field  performance. 

RATING: 


1.  Storage  and  Retrieval  of  Student's  Pre-  and  Post-Course 

Attitudes  -  for  example,  student's  attitudes  toward 
training,  materials,  or  methods.  RATING:  _ 

2.  Storage  and  Retrieval  of  Changes  in  Interests  or  Attitudes 
During  Training  -  for  example,  periodic  questionnaires  to 
assess  interests  in  the  training  itself,  or  the  media. 

RATING: 


II.  STUDENT  ASSIGNMENT  (IN  SELF-PACED  COURSES)  -  Use  of  the  computer  to 
assign  trainees  to  lessons,  tests,  or  other  instructional  alternatives. 
The  assignments  can  be  based  on  considerations  of  lesson  content,  avail¬ 
able  resources,  student  characteristics,  or  student  performance  data. 
Students  receive  their  next  assignment  immediately  after  completing 
their  preceding  lesson,  and  training  resources  are  scheduled  for 
optimal  usage. 

A.  Assignment  to  Alternative  Course  Versions  -  Course  versions 
which  differ'Ts  to  content  ( perhaps  for  different  shredouts  of 
a  specialty,  or  to  test  and  implement  new  training  materials) 
can  be  established.  A  student  enrolled  in  a  particular  course 
version  will  be  assigned  only  the  materials  for  that  version. 

RATING: 


B.  Determination  of  the  Order  of  Assignments  -  The  computer  takes 
Thtcf Account  various  classes  of  information  in  determining  what 
to  assign  "ext  to  a  trainee.  The  information  considered  can 
include,  for  example,  work  the  trainee  has  completed  to  date, 
the  trainee's  rate  of  nrogress  to  date,  availability  of  instruc¬ 
tional  resources,  and  requirements  to  assemble  a  team  of 
trainees  for  a  training  task.  RATING:  _ 

1.  Order  of  assionments  based  on  lessons  completed  t.o  date. 

RATING:  _ 

Order  of  assignments  based  on  performance  (time  and 
scores)  on  earlier  lessons.  RATING: 

3.  Order  of  assignments  based  on  availability  of  resources. 

RATING: 

3.  Order  of  assignments  hast'd  on  considerations  of  assembling 
a  team  of  trainees  for  a  team  task.  RATING: 
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5.  Order  of  assignments  determined  bv  allowing  instructor  to 

pick  from  available  lessons.  HATING: 

C.  Assignment  to  Alternate  Lessons  -  Provides  for  student  assign¬ 
ment  to  available  alternate  media  (e.g.,  printed  vs.  audio¬ 
visual  media)  or  alternate  versions  (e.q.,  easy  versus  difficult). 

RATING: 

1.  based  on  Instructor  Selection  -  instructors  select  alter¬ 
nates  based  on  their  assessment  of  the  student's  needs. 

RATING: 

L’.  based  on  Student  Selection  -  students  select,  from  lists  of 
avaTTable  TTternate  media  or  lessons,  the  particular  one(s) 
the  prefer.  RATING: 

d.  !idsjpd__cin  Resource  Availability  -  computer  program  makes  the 
selection  of  a  particular  media  nr  lesson  on  the  basis  ot 
ava  i  labi 1 i ty .  RATING: 

A.  based  on  N i thi n-Course  Student  Performance  -  coirniter  p ro¬ 
il  ram  selects  a  particular  media*  or  lesson  on  the  basis  of 
the  student's  prior  test  scores,  times-to-cemplete,  or 
wi th in-course  motivation/ i uteres t . 

RA i 1 NG : 

!>.  Rased  on  Pre-Course  Student  Characteristics  -  computer  pro¬ 
gram  selects  a  particular"  media  or  lesson  on  the  basis  of 
the  student's  general  aptitudes,  abilities,  attitudes, 
interests  or  other  [ire-course  data.  RATING: 

6.  Coinjuiter  Selections  based  on  Logical  Rules  -  simple 

"if.. then"  logical  rules  arc  used  in  the  selection  of  media 
or  lessons.  RATING: 

7.  Computer  Selections  Based  on  Performance  Predictions  - 
student  characteristics  and  performance  are  used  to  predict 
which  media  or  lesson  is  best  for  a  given  student. 

RATING: 

D.  Assignment  to  Alternative  Remediation  Activities  -  Provides  for 
assignments  to  available  alternate  remediation  materials  and 
media  (e.o.,  CA1  reviews,  instructor  tutorials,  etc.). 

RATING: 

1.  Lessons  -  the  computer  assigns  each  student  to  tin'  best 

ava liable  remediation  alternative  for  second  and  subsequent 
attempts  to  master  lesson  objectives.  RATING: 
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2.  Lesson  Groups  -  the  computer  assigns  the  best  available 
remediation  alternatives  to  each  student  who  Is  having 
difficulty  mastering  the  objectives  In  a  related  group 
of  lessons.  RATING: 


3.  Block  or  Course  Level  -  the  computer  assigns  the  best  avall- 
able  remediation  alternatives  to  each  student  who  is  having 
difficulty  mastering  block  or  course  level  objectives. 

RATING: 


E.  Test  Selection  and  Assignment  -  Different  forms  of  the  tests 
can  lie  selected  and  assigned  by  the  compute*-.  The  computer 
can  also  determine  if  a  particular  trainee  bould  take  an  on¬ 
line  ( CA I )  or  an  off-line  (paper  and  pencif.'  'orsion  of  a  test. 

RATING: 


1.  Assign  the  different  forms  of  a  test  at  random. 

RATING: 


2,  Exclude  versions  that  were  taken  by  the  trainee  on  an 

earl ier  attempt.  RATING:  _ 

3.  Computer  determines  whether  to  assign  on-line  or  off-line 

version  of  test.  RATING: 


Assignment  to  Supplementary  Skill  Training  at  Course  Entry  - 
Provides  for  assigning  those  students  identified  as  having 
deficiencies  in  critical  entry  or  learning  skill  areas  to 
supplementary  or  remedial  skill  training.  Students  would  be 
assigned  in  these  areas  prior  to  beginning  training  or  early 
in  the  training  process  by  specialized  materials  or  instruc¬ 
tional  procedures.  RATING:  _ 

1.  Based  on  Critical  Entry  Skill  Assessment  -  for  example, 

students  would  be  assigned  to  materials  to  improve  their 
reading  skills  or  mathematical  skills  prior  to  beginning 
training.  RATING:  _ 

2.  Based  on  Deficient  Learning  Skills  Assessment  -  for  example, 

students  would  be  assignedto  materials  to  remedy  defi¬ 
ciencies  in  general  study  habits  and  skills,  including  con¬ 
centration  skills,  memorization  skills,  and  test  taking 
skills.  RATING: 


G.  Additional  Resource  Management  Considerations  -  The  system  can 
determine  wh fell  resources  are  needed" "by  a  lesson ,  if  they  are 
available,  and  where  they  are  located.  Managed  resources  can 
include  classroom  spaces,  lesson  materials,  media  devices,  work 
stations,  simulators,  and  other  training  equipment.  Students 
can  be  assigned  in  a  manner  that  will  even  out  the  distribution 
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of  students  throughout  the  course  and  that  will  attempt  to  keep 
as  many  assignment  options  available  for  the  student  as  possible. 

RATING: 

• 

1.  Student  assignment  to  the  learning  center  or  classroom 

with  the  most  room  (other  instructional  considerations 
being  equal ) .  RATING:  __ 

2.  Student  assignment  to  blocks  of  instruction  to  achieve 

an  optimal  distribution  among  blocks  (other  instructional 
considerations  being  equal).  RATING:  _ 

3.  Capability  to  assign  students  to  resources  that  are  outside 
their  own  learning  center  or  classroom  (that  is,  capability 
to  share  resources  among  several  classrooms). 

RATING: 

III.  STUDENT  PROGRESS  MANAGEMENT  -  Computer-based  capabilities  to 
support  the  management  of  a  trainee's  progress  through  a  self-paced 
training  process.  These  capabilities  range  from  simple  scheduling  to 
sophisticated  progress  prediction  and  management.  Both  the  student  and 
instructor  know  how  the  student  is  progressing  at  all  times  -  on 
Schedule,  ahead,  or  behind. 

A.  Scheduling  Student  Entries  into  the  Course  -  Notifying  the 

student  squadron  or  other  authority  as  to  which  students  can 
start  the  course  each  day  based  on  available  space  in  the 
starting  block  or  blocks.  RATING: 

B.  Graduation  Forecasting  -  Predicting,  on  the  basis  of  the  stu- 

dent's  performance  in  the  course,  the  date  on  which  lie  will 
complete  the  course,  barring  subsequent  absences.  Predicted 
graduation  dates  can  be  shown  on  the  Learning  Center  roster 
and  special  reports  can  list  the  students  predicted  to  graduate 
within  a  specified  number  of  days.  Required  out-processing 
activities  can  be  scheduled,  reports  for  the  Base  Personnel 
office  can  be  prepared,  or  the  System  can  be  directly  inter¬ 
faced  to  the  Personnel  Computer  System.  RATING:  _ 

1.  Predicted  graduation  date  shown  on  roster  RATING:  _ 

2.  Predicted  graduation  dates  within  a  specified 
number  of  days  are  available  by  special  report 

RATING:  _ 

3.  System- genera ted  reports  to  Base  Personnel  Office 

RATING:  _ 

4.  Direct  interface  to  Base  Personnel  Computer  System 

RATING: 
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5.  System  scheduling  of  out-processing  activities 

RATING: 


C.  Monitoring  and  Management  of  Student  Progress  to  a  Target  - 
Establishing  a  target  completion  date,  for  the  whole  course  or 
for  Individual  blocks  for  each  student  on  the  basis  of  pre¬ 
course  characteristics  (e.g.,  ASVAB  or  Pre-course  test  scores) 
or  performance  In  the  early  blocks  of  the  course.  Both  the 
student  and  the  instructor  will  receive  feedback  on  the  stu¬ 
dent's  progress  relative  to  his  target.  Targets  can  be  made 
relative  to  either  average  actual  course  length  or  the  POI 
course  length.  RATING:  _ 

1.  Targets  and  management  printouts  addressing  the  whole 
course.  RATING: 


2.  Targets  and  management  printouts  addressing  individual 
blocks.  RATING: 


3.  Use  of  pre-course  data  In  establishing  target  dates. 

RATING: 


4.  Use  of  within-course  data  in  establishing  target  dates. 

RATING: 


5.  Targets  are  established  relative  to  average  course  length. 

RATING: 


6.  Targets  are  established  relative  to  POI  course  length. 

RATING: 


IV.  SUPPORT  FOR  GUIDANCE  AND  COUNSELING  -  Computer-based  capabilities 
to  support  student  guidance  and  counseling  by  instructors  and  course 
management  personnel.  These  capabilities  can  be  based  on  identifying 
entry  skill  deficiencies,  deficiencies  in  within-course  performance,  or 
both.  Instructors  can  quickly  determine  which  students  need  counseling, 
and  can  retrieve  from  the  computer  the  information  needed  to  give  mean¬ 
ingful  guidance  to  the  student. 

A .  Identification  of  Students  Needing  Special  Attention  at  Course 
Entry  -  Provides'for  the  identification  and  reporting  of  stu- 
dents  expected  to  be  proficient  or  deficient  with  respect  to 
mastery  of  course  or  training  objectives.  This  identification 
and  reporting  is  initiated  at  the  beginning  of  the  training 
process  to  focus  guidance  and  counseling  functions  on  those 
students  most  in  need  of  special  trainina  procedures. 

RATING: 
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1.  Students  Flagged  on  Pre-determined  Entry  Variables  -  for 
example,  a  list  of  students  measured  as  proficient  or  as 
deficient  on  precourse  assessments  can  be  printed. 

RATING:  _ 

2!.  Proficient  and  Deficient  Student  Scores  Printed  -  besides 
listing  students  expected  to  be  proficient  or  deficient, 
their  scores  on  selected  critical  entrv  skill  variables 
(e.g.,  reading,  math,  study  skills)  can  also  bo  printed. 

RATING: 


3 .  Scores  for  All  Students  on  Critical  Entry  Variables  Printed  - 
a  report  which  includes  the  "scores  of"  all  students  on  tluiso 
variables  identified  as  critical  entry  skills,  flagging 
those  student  scores  which  are  deficient.  Allows  extra 
capability  to  watch  students  who  may  be  marginal. 

RAT  Hit! : 

4.  Proficient  and  Deficient  Student  Predicted  Completion  Times 

Printed  -  in  addition  to  the  foregoing  information,- The 
predicted  training  completion  times  can  be  printed  for  those 
students  expected  to  be  proficient  or  deficient  in  mastering 
training  objectives.  RATING: 

Identification  of  Marginal  Student  Performance  Wi thi n-Course  - 

Provides  for  the  identification  and  reporting  of  students 

expected  to  have  difficulty  mastering  training  objectives. 

Iaenti fication  and  reporting  are  initiated  during  training  to 

direct  special  help  to  those  students  most  in  need  of  it. 

RATING:  _ 

1 .  Students  Flagged  on  Pre-determined  Time  and  Score  Variables  - 
for"  example,  a  list  of  those  students  exceeding  predetermined 
training  time  cut-offs,  or  scoring  below  predetermined 
criterion  levels  on  training  objectives,  can  be  printed. 

RATING: 

2.  Times  and  Scores  for  Marginal  Students  Printed  -  for 

example,  besides  listing  students  whose  performance  in 
training  is  marginal,  their  actual  time  and  score  data  can 
also  be  printed.  RATING: 

3.  Students  Identified  for  Possible  Elimination  from  Training  - 

of  students  listed  as  marginal,  those  whose  time  cr  score 
data  are  below  course-established  minimums  for  acceptable 
performance  would  be  flagged  as  possible  candidates  for 
elimination.  RATING: 


4 .  Students  Identified  for  Specialized  Remedial  Training  - 

students  identified  as  marginal  on  the  basis  of  their  train- 
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ing  t i n»es  or  scores  receive  specialized  remedial  training 
assignments  (e.g.,  to  special  instructor  tutorials, 
remediation  sessions  out-of-class,  or  special  skill  training 
for  remedying  learning  or  stuay  skills  weaknesses). 

RATING: 


V.  COMPUTER-AIDED  INSTRUCTION  (CAI)  -  Lesson  materials  are  stored  in 
the  computer,  and  on  request  are  displayed  to  a  trainee  for  study.  The 
stored  materials  can  include  text,  questions,  and  graphics.  The  trainee 
may  interact  with  t lie  computer  through  a  keyboard,  by  pointing  (touch 
panel),  or  with  a  light  pen  (pointing  to  or  drawing  on  the  display 
screen).  Student  interaction  can  range  from  "turning  the  pages"  on  up 
through  very  sophisticated  exchanges  -  for  example,  in  simulated  trouble¬ 
shooting,  various  information  can  be  requestea  from  the  computer,  and 
various  solutions  can  be  tested. 

A.  CAI  Applications  -  There  are  five  different  ways  in  which  CAI 
can  be  used  in  Air  Force  training:  drill  and  practice  - 
practice  with  correction  and  guidance  of  basic  skills  and 
knowledges  learned  via  other  media;  tutorial  -  the  use  of  CAI 
for  particular  whole  lessons  which  are  particularly  difficult; 
simulation  -  use  of  CAI  to  simulate  equipment  or  processes  and 
to  teach  students  to  use  the  equipment  or  follow  the  process; 
review  and  remediation  -  use  of  CAI  to  quickly  review  the  con¬ 
tent  of  a  block  prior  to  a  block  test  or  to  remediate  over 
failed  objectives  following  a  block  test  failure;  and  study 
skills  -  use  of  CAI  to  teach  specific  study  skills  to  students 
who  are  deficient  in  these  areas.  RATING: 


1.  Drill  and  Practice  CAI  in  your  training  area. 

RATING; 


Tutorial  CAI  in  your  training  area.  RATING: 


3.  Simulation  CAI  in  your  training  area.  RATING: 


4.  CAI  for  Block  Review  prior  to  the  block  test. 

RATING 


5.  CAI  for  Block  Remediation  after  a  failure.  RATING 


6.  CAI  for  teaching  study  skills.  RATING:  ____ 

B.  CAI  Capabilities  -  Regardless  of  the  particular  applications, 
the  sophistication  of  the  CAI  capability  can  vary  from  lean  to 
rich.  The  most  basic  CAI  system  would  present  text,  and  allow 
input  to  multiple-choice  questions  via  a  typewriter  keyboard. 
The  CAI  programs  would  contain  the  capability  to  adapt  to  the 
student's  responses  but  they  would  not  access  other  information 
about  the  student  stored  in  the  computer.  A  richer  CAI  system 
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will  support  graphics  (line  drawings)  as  well  as  text,  allow 
students  to  enter  constructed-responie  (open-ended)  answers,  to 
respond  by  touching  the  screen  (using  a  touch-sensitive 
surface),  or  with  a  light-sensitive  pointer  (light  pen), 
control  a  supplementary  slide  or  film-strip  presentation, 
control  an  audio  tape,  and  retrieve  other  information  about 
the  student  from  the  data  base  to  allow  more  adaptive 
instruction. 

1.  Basic  CAI  -  presenting  text,  and  allowing  input  to  multiple 

choice  questions  from  a  keyboard.  Has  the  capability  to 
adapt  to  the  answers,  but  does  not  access  other  information 
about  the  student.  RATING:  _ _ 

2.  Adoing  graphics  capability  to  the  basic  CAI. 

RATING:  _ 

3.  Adding  constructed  response  test  items  (English 

answers)  to  the  basic  CAI.  RATING:  _____ 

4.  Adding  touch-sensitive  or  light  pen  capability. 

RATING;  _ _ 

s .  Adding  audio  tape  control  to  basic  CAI  -  can 
present  voice  accompaniment  during  selected 
parts  of  the  lesson.  RATING:  _ 

6.  Adding  computer  control  over  a  slide  or  filmstrip 
projector  -  can  present  supplementary  visual  material 
to  accompany  the  CAI  display  in  designated  parts  of 

the  lesson.  RATING:  _ 

7.  Providing  additional  student  information  from  the 
computer  data  base  -  for  examnle  ASVAR  or  AFQT  scores, 
realing  ability  scores,  or  penc.-mance  on  an  earlier 
lesson  -  to  increase  the  soohi stication  of  adapting 

the  CAI  oroar3m  to  the  student's  abilities  and  interests. 

RATING:  _ 

On-line  Learnjjir  Process  or  Specialized  Skill  Training  -  Pre- 
sentation  of  specialized  training  in  a  highly  "Interactive, 
computer-assisted  format  to  either  students  or  instructors.  In 
the  case  of  students,  the  on-line  materials  would  be  designed 
for  learning  skill  training  (e.g.,  study  habits  and  skills, 
test-taking  skills,  memorization  skills)  for  those  students 
identified  as  needing  this  training.  In  the  case  of  instruc¬ 
tors,  the  on-line  materials  would  be  designed  to  train 
diagnostic,  tutorial,  or  other  counseling  skills  for  those 
instructors  who  volunteer  or  are  selected  for  tins  training. 

RATING: 


1.  On-line  Student  Skill  Training  -  for  example,  for  students 
identified  as  deficient  in  critical  entry  skills  or  as 
having  marginal  wl thin-course  performance,  assignment  to 
specialized  on-line  materials  would  be  provided. 

RATING:  _ 

2.  On-line  Instructor  Skill  Training  -  'for  example,  for  in¬ 
structors  who  volunteer  or  are  selected  for  specialized 
skill/role  training,  assignment  to  specialized  on-line 
training  materials  and  sequencer  would  be  provided. 

RATING:  _ 

I.  CMI  DATA  RASE  AND  CAI  MATERIALS  PRODUCTION  AND  MAINTENANCE  -  To 
erform  any  training  functions,  the  computer  must  be  informed  about  the 
rainees  (name,  social  security  number,  course,  etc.),  the  course 
t.ructure  (tests,  lessons,  resources,  allowable  sequences  of  lessons, 
tc.),  and  student  performance  (lessons  completed,  test  scores,  block 
ompletion  times,  etc.).  The  CMI  Data  Base  refers  to  entry,  storage, 
nd  maintenance  of  this  information. 

In  the  case  of  Comouter  Assisted  Instruction  (CAI),  all  of  the 
esson  information  (text,  questions,  graphics,  branching  instructions, 
tc.)  must  be  stored  in  the  computer.  CAI  Materials  Production  and 
aintenarce  refers  to  entry  storage  and  maintenance  of  this  information. 

A.  CMI  Course  Definition  Data  Rase  Editing  -  Used  by  course  person¬ 
nel  to  enter  the  information  that  describes  the  course  structure 
to  the  computer.  This  editing  should  be  easy  to  accomplish  and 
easy  to  modify.  It  can  be  an  off-line  card  based  system  (course 
personnel  describe  the  course  structure  on  paper,  and  later 
receive  a  printout  listing  what  has  been  entered  into  the  data 
base),  or  can  run  on-line  (course  personnel  at  a  computer 
terminal  enter,  display,  and  change  the  information  directly). 

To  facilitate  use,  the  on-line  system  can  use  graphics  to  dis¬ 
play  the  course  structure  (the  allowable  paths  through  the 
lesson  materials). 

1.  Making  the  system  on-line,  rather  than  the  off-line 

card  based  approach.  RATING:  _ 

2.  Using  graphics  to  display  the  course  structure,  rather 

than  relying  completely  on  words.  RATING:  _____ 

R .  Auto matic  Val  idati o n  of  Course  Data  Rase  Integrity  -  Capability 
to  check  automatically-" if  all  required  data  base  records  to 
describe  the  course  are  present,  and  consistent.  When  a  new 
course  or  course  version  is  implemented,  or  an  existing  one  is 
changed,  all  of  the  lessons,  tests,  and  blocks  must  be  in  the 
data  base.  This  capability  will  automatically  check  for 
completeness.  RATING:  _ 
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Editors  for  Production  of  On-line  Tests  -  If  the  System  includes 
We~capaFnity  for  on-line  testing,  a  means  must  be  provided  to 
enter  the  test  questions  and  their  alternatives  into  the 
computer  system's  data  base  -  a  Test  Item  Editor.  The  simplest 
form  of  the  Editor  would  allow  the  author  to  type  in  text.  Add¬ 
itional  features  would  format  the  test  question  for  the  author, 
allow  the  author  to  decide  whether  or  not  to  randomise  the 
presentation  order  of  test  items  and  of  alternatives  within 
test  items,  control  the  number  of  attempts  the  student  could 
make  on  each  item,  and  enter  feedback  messages  to  follow 
correct  and  incorrect  answers.  RATING:  _ 

1.  Adding  automatic  formatting  for  the  entry  of  test 

questions.  RATING:  , _ 

2.  Adding  author  control  over  whether  order  of  test 
items  and  alternatives  will  be  randomi red. 

RATING:  _ 

3.  Adding  author  control  over  the  number  of  attempts 

allowed  on  a  question.  RATING:  _ 

4.  Adding  feedback  messages  for  correct  and  incorrect 

answers,  at  the  author's  option.  RATING: 

CAI  Authoring  Fdi tor  -  If  the  System  includes  CA1,  there  must 
be  a  means  of  producing  the  CAI  materials  or  programs.  The 
traditional  approach  is  to  have  CAI  authors  learn  an  authoring 
language,  and  use  it  in  writing  their  lessons.  An  alternative 
is  to  have  CAI  lessons  written  by  an  author/prograniner  team. 

These  approaches  require  practice  and  experience.  Another 
alternative  is  to  provide  a  CAI  Authoring  Editor  which  structures 
the  author's  task  for  him,  provides  standardized  formats  for 
text  frames  and  questions,  and  eliminates  the  need  for  any  pro¬ 
gramming  ability.  The  program  format  would  not  bo  rigid  and 
alternative  presentation  approaches  can  bo  provided  by  the 
Editor.  Instructions  for  using  the  editor  can  be  in  a  manual 
and  in  addition,  instructions  can  be  displayed  in  the  Editor 
program  itself  through  a  HELP  request.  The  HELP  information 
can  refer  just  to  the  operation  of  the  Editor  or  can  also 
provide  guidelines  for  good  instructional  practices. 

1.  Providing  a  CAI  Authoring  Editor,  rather  than  requiring 
CAI  authors  to  learn  the  prograirrii ng  language  or  work 

with  a  programner  to  construct  lessons.  RATING:  _ 

2.  Providing,  in  the  CA!  Authoring  Editor,  automatic 
formatting  for  text  frames  and  for  questions. 

RATING: 
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3.  Providing,  in  the  Editor,  automatic  structuring  of 
the  sequence  of  frames  and  questions  in  the  CAI 
presentation.  RATING: 


4,  Providing  capability  to  copy  text  frames,  questions, 
and  graphics  from  other  CAI  lessons.  RATING: 


5.  Providing  a  HELP  request  to  display  instructions  foi 
using  the  Editor  instead  of  relying  entirely  on  an 
instruction  manual .  RATING: 


6.  Adding  guidelines  for  good  instructional  practices 
as  part  of  a  HELP  request  at  the  terminal.  RATING: 


E .  Use  of  Test  Item  and  CAI  Authoring  Editors  for  Off-Line 

Materials  -  If  "Test  Item  and  CAI  Authoring  Editors  are  avail¬ 
able,  it  may  be  useful  to  use  them  for  producing  tests  and 
instructional  materials  which  are  intended  for  use  off-line, 
e.g.,  regular  programmed  texts.  The  Editors  can  be  used  to 
write,  review,  and  revise  the  tests  and  materials  and  the  final 
product  can  be  printed  out  ready  to  be  reproduced.  If  the 
number  of  copies  reauired  is  not  too  great,  multiple  copies  can 
be  printed  to  avoid  the  need  for  reproduction.  Finally,  small- 
group  tryouts  could  be  conducted  on-line  (like  CAI)  to  provide 
more  detailed  information  on  students'  reactions  to  the 
materials.  RATING: 


1.  Using  the  Test  Item  Editor  to  construct  and  produce 

off-line  tests.  RATING: 

2.  Using  toe  CAI  Authoring  Editor  to  construct  and 

produce  off-line  lesson  materials.  RATING: 

3.  Using  Editors  for  review  and  revision  of  tests  and 

lesson  materials.  RATING: 

4.  Printing  out  a  master  copy,  ready  for  reproduction,  of 

tests  and  materials  which  have  been  written  using  the 
Editors.  RATING: 

5.  Printing  out  multiple  copies  of  tests  and  materials, 

in  order  to  bypass  reproduction  for  small  quantities. 

RATING: 


G.  Providing  capability  for  conducting  on-line  small 
group  tryouts  of  materials  and  tests.  RATING: 


F.  Use  of  On-Line  Editors  for  Production  and  Maintenance  of  Course 
Documents  -  An  Editor  like  those  described  fur  te^L  Items  and 
CAI  authoring  could  be  used  for  producing  course  documents  such 
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as  the  POI,  course  charts,  lesson  outlines,  etc.  The  documents 
would  then  be  available  at  any  time  through  the  Editors  for 
review  or  revision  and  copies  could  be  printed  out  as  needed. 


RATING: 


VII.  INFORMATION  RETRIEVAL  AND  REPORTS  -  In  a  computer-based  training 
system,  large  amounts  of  information  about  trainees,  courses,  course 
materials,  tests  and  test  items,  and  other  parts  of  the  system  are 
stored,  and  are  available  for  periodic  {hourly,  daily,  weekly, 
monthly....)  analyses  and  reports.  These  functions  are  intended 
principally  for  instructors,  managers,  and  administrators.  Thny  can  be 
valuable  to  instructors  who  must  provide  management  and  guidance  to 
students.  They  also  can  be  valuable  in  providing  quality  control  over 
students  and  materials,  in  improving  the  management  and  administration 
of  training,  and  in  evaluating  and  improving  all  aspects  of  development 
and  implementation  of  training  materials  and  programs. 

A.  Reports  for  Instructors  -  Large  amounts  of  data  can  be  collected 
ana  made*  a  variable  toTnstructors,  but  the  reports  should  be 
carefully  structured  to  meet  the  needs  of  the  instructors.  The 
reports  may  Include:  classroom  rosters,  summaries  for  specific 
parts  of  the  course.  Individual  history  reports,  and  sii.na-ies 
for  the  classroorj.  RATING: 


1.  Learning  Center  or  Classroom  Rosters  -  Rosters  can  be  dis¬ 
played  or  printed  to  show  many  different  kirds  uf  'nf*«ma- 
tion  about  each  of  the  students  in  a  classroom  or  learning 
center.  The  basic  information  would  include  social  security 
numbers,  names,  carrel  (or  position)  numbers,  absences, 
current  block,  and  rate  of  progress.  RATING:  _ 


2.  Individual  Block  Progress  Reports  -  The  computer  system 
can  allow  review  of  data  by  instructors,  supervisors, 
counselors,  etc.,  via  local  terminals.  Progress  reports 
can  provide  data  for  in-course  counseling  and  can  become 
economical  replacements  for  some  of  the  current  manually 
prepared  forms  which  accompany  student  records  throughout 
the  course  and,  in  some  cases,  beyond  graduation.  An 
Individual  Block  Progress  Report  will  summarize  student 
progress,  times  and  scores,  and  other  pertinent  information. 

RATING: 


Information  included  in  the  Wock  progress  reports  can 
include  some  or  all  of  the  following: 

a)  lessons  completed  by  the  student;  RATING: 


b)  numbers  of  attempts,  and  scores  on  each 

attempt,  on  each  lesson;  RATING: 
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c)  objectives  failed  during  the  block; 


RATING; 


d)  time  spent  on  each  lesson  and  on  the 

block;  RATING: 


e)  amount  of  time  student  Is  ahead  or  behind 

schedule;  RATING: 


f)  amount  of  homework  student  has  accomplished. 

RATING:  _ 

3.  Individual  Student  History  Reports  -  The  system  can  provide 
a  large  amount  oT  additional  data  about  an  Individual 
student.  This  can  include  the  student's  scores  on  precourse 
aptitude  and  Interest  measures,  scores  on  critical  entry 
skill  tests,  time  spent  in  block  remediation  following  a 
block  failure,  and  absence  times  and  reasons  for  absence. 

RATING:  _ 

Information  in  the  Individual  Student  History  Report  can 
Include  some  or  all  of  the  following: 

a)  attitude  and  interest  information;  RATING:  _ 

b)  scores  on  critical  entry  skills;  RATING:  _ 

c)  time  in  block  remediation  following  a  block 

failure;  RATING:  _ 

d)  lesson  objectives  that  were  failed  more  than 

twice;  RATING: 


e)  time  absent  in  each  block,  and  reasons  for 

absences.  RATING: 


4.  Additional  Reports  on  the  Student  - 

a)  Absence  Report,  to  replace  the  manual  absence 
report;  RATING: 


b)  Homework  Summary  Report,  to  list  lessons  accomplished 
as  homework  and  to  summarize  homework  time  expended; 

RATING:  ___ 

c)  On-line  display  of  student  data  -  the  capability  to 

call  up  and  display  any  of  the  available  student 
information.  RATING: 
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B.  Training  Management  Reports  -  Reports  such  as  these  are  intended 
for  use  by  managers  (Shift  Supervisors  and  above)  in  monitoring 
training  activities.  Potential  reports  include  number  of  stu¬ 
dents  awaiting  training  and  how  long  they  have  been  waiting, 
number  of  trainees  in  each  block  or  each  classroom  on  each  base, 
the  distribution  of  completion  times  for  blocks  and  the  course 
itself,  differences  in  trainee  performance  between  parallel 
classrooms  or  bases,  the  extent  to  which  instructional  resources 
such  as  carrels,  equipment,  and  training  devices  are  being 
utilized,  and  summaries,  by  block  or  other  unit,  of  trainee  per¬ 
formance. 

1.  Report  on  Students  Awaiting  Training;  RATING:  _ 

2.  Report  of  number  of  students  currently  in  each  block, 

classroom,  or  on  each  base;  RATING:  _ 

3.  The  distribution  of  completion  times  for  individual 

blocks;  RATING:  _ _ 

4.  The  distribution  of  course  completion  times; 

RATING:  _ 

5.  Differences  in  student  performance  across  parallel 

learning  centers,  classrooms  or  bases;  RATING:  _ 

6.  Reports  of  instructional  resource  (facilities, 

training  devices,  etc.)  utilization;  RATING:  _ 

7.  Report  of  utilization  and  performance  of 

instructors.  RATING: 


8.  Sunmary  Reports  of  student  completion  times  and 

final  grades,  by  block.  RATING: 


C.  Course  Evaluation  Sunmary  -  Suronarizes  student  performance  on 
incfi vidual  lessons  within  blocks  and  on  the  block  itself.  It  is 
intended  for  use  in  small -group  tryouts  of  new  materials  and 
monitoring  student  performance  on  established  materials.  The 
Course  Evaluation  Summary  can  contain  a  variety  of  features. 


listed  below.  RATING: 

1.  First-attempt  lesson  and  block^test  scores.  RATING: 

2.  Final  lesson  and  block  test  scores.  RATING: 

3.  Mean  and  standard  deviation  of  first-attempt  lesson 

and  block  test  scores.  RATING: 


4.  Mean  and  standard  deviation  of  final  lesson  and 

block  test  scores.  RATING: 


5.  Mean  and  standard  deviation  of  first-attempt  lesson 

and  block  times.  RATING: 

6.  Mean  and  standard  deviation  of  lesson  and  block 

completion  times.  RATING: 


7.  Separate  score  and  time  data  for  alternative  modules 
within  lessons.  RATING: 


8.  Score  data  broken  out  by  objective  within  lessons. 

RATING: 


9.  First-attempt  lesson  failure  rates.  RATING: 

10.  First-attempt  objective  failure  rates.  RATING: 


11.  Correlations  showing  the  relationship  between 
predicted  and  actual  lesson  and  block  times 
and  scores.  RATING: 


12.  Summarized  data  on  lessons,  modules  and  objectives 
with  first-attempt  failure  rates  above  a  certain 
percentage.  RATING:  _ 

Test  Item  Evaluation  Report  -  Provides  detailed  information 
the  characteristics  of  Mock  and  lesson  tests.  It  cou’d  con 
tain  a  variety  of  features,  listed  below.  RATING: 


1.  The  number  (or  percentage)  of  trainees  selecting  each 
alternative  answer  to  test  questions.  RATING: 

Percentage  of  trainees  answering  each  question 
correctly.  RATING: 

3.  Average  test  score  of  trainees  selecting  each 

alternative  on  each  question.  RATING: 

4.  Flagging  test  questions  missed  by  over  70  percent 

or  by  none  of  the  trainees.  RATING: 

5.  I tem- remainder  correlations  for  each  question. 

RATING: 


6.  Means  and  standard  deviations  of  scores  for  each 

objective  and  for  each  full  test.  f<AT I NG : 


7. 


RATING: 


Distribution  of  scores  on  the  full  test. 


G.  Alpha  reliability  coefficient  for  each  objective 
and  for  the  full  test.  RATING: 


9.  Criterion-referenced  reliability  coefficients  for 
each  objective  and  for  the  full  test.  RATING: 


E.  CAI  Evaluation  Data  -  Since  CAI  involves  frequent  interaction 
with"  the  student,  very  detailed  evaluation  data  are  available 
from  CAI  programs.  The  most  frequent  use  of  this  information 
would  be  during  formative  evaluation  (small -group  tryouts)  of 
the  programs.  A  variety  of  capabilities  are  listed  below. 

1.  Percentage  of  students  selecting  each  alternative  in 

each  question  frame.  RATING: 

2.  Percentage  of  students  selecting  each  alternative 

in  each  question  frame  on  their  first  try,  their 

second  try,  etc.  RATING:  _ 

3.  The  means  and  standard  deviations  of  the  time  to 

respond  to  question  frames  and  the  time  to  read 
text  frames.  RATING: 


4.  Lists  of  unanticipated  responses  to  constructed 

response  questions.  RATING: 


5.  Lists  of  open-ended  student  comments  about  the  CAI 
program.  RATING: 


6.  Detailed  records  of  each  student’s  path  through  a 

CAI  program.  RATING:  _ 

« 

7.  Summary  statistics  on  major  units  of  the  program, 
such  as  time  to  complete  and  cumulative  score. 

RATING:  _ 

F.  Courseware  development  Management  Reports  -  When  editors  arc  used 
for  developing  on-line  tests,  CAt  materials,  or  tests  and 
materials  to  be  used  off-line,  the  System  can  capture  information 
about  authors'  work  on  the  Editors  and  the  status  of  the 
development  effort.  This  information  could  then  he  summarized 
in  management  reports  for  monitoring  materials  production, 
review,  revision,  and  Implementation,  The  reports  could  include 
information  such  as  the  person  responsible  tor  developing, 
reviewing,  or  revising  n  lesson,  how  long  they  have  been  work¬ 
ing  on  it,  how  inui  h  of  the  work  has  been  completed,  and  expected 
completion  date.  RATING:  _ 
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Performance  Forecast  Reports  -  Provides  field,  00T,  or  technical 
training  personnel  with  training  Information  which  describes 
current  student  performance  levels  or  predicts  how  Individual 
students  will  perform  in  specific  training  areas. 

RATING: 


1*  PrgjServIce  Education  and/or  Vocational  Training  -  available 
individual  data  relevant  to  prior  education  or  training. 

RATING:  _ 

2.  P^e-Service  Occupational/Job  Experience  -  data 
which  reflects  prior  relevant  job  experience. 

RATING: 


3-  Enlistment/Classification  Data  -  data  available  in  AF 
personnel  records  (e.g.,  ASVAB  scores,  AFOQT  or  AFQT 
scores).  RATING: 


In-Service  Training  Performance  Data  -  for  example,  for 
FTD  or  OJT  managers,  a  report  of  Individual  student 
technical  training  or  in-service  training  performance 
data  (e.g.,  scores,  times-to-complete) .  RATING: 


5.  AF  Specialty-Related  Deficiencies  -  for  example,  student 
data  on  deficiencies  in  skills,  experiences,  or  abilities 
relevant  to  that  particular  individual's  specialty. 

RATING: 


6.  idents'  Predicted  Adaptability,  Performance  and 

I i egression  in  Operational  field  Assignments  -for  example, 
studerrts^-  in-service  and  pre-service  data  could  be  used  to 
provide  reports  of  predicted  field  assignment  adoptability, 
performance  and  pregression,  RATING: 


Special  Purpose  Reports  -  No  matter  how  extensive,  no  set  of 
standard  reports  will  always  be  enough  to  answer  all  management 
and  evaluation  questions.  It  is  useful,  therefore,  to  have  an 
open-ended  information  retrieval  capability  that  can  br  used  to 
answer  unexpected  questions  about  student  and  System  performance 
A  minimal  capability  would  permit  the  user  to  define  the 
variables  which  he  wishes  to  look  at,  and  the  ty  w»s  of  students 
and  time  periods  from  which  the  data  are  to  be  drawn,  [he  out¬ 
put  would  consist  of  the  number  of  cases  found,  and  the  mean, 
standard  deviation,  and  range  for  each  included  variable.  Such 
a  program  could  be  run  interactively,  in  a  minute  or  so,  and  the 
results  displayed  on  the  user's  screen  or  it  could  bo  run  as  a 
background  job,  in  a  half-hour  or  so,  and  the  results  printed 
out. 


1.  Utility  of  basic  Data  Retrieval  Program  w’ ti.  orij 

background  capability.  RATING:  _ 

2.  Utility  of  adding  interactive  {as  well  as  back¬ 
ground)  capability.  RATING:  _ 

3.  The  basic  data  retrieval  program  could  also  be  inter¬ 
faced  to  one  of  the  standard  packages  of  statistical 
programs  (for  example,  SPSS,  or  BMD).  This  would  permit 
extensive  analysis  of  student  data  and  the  capability 

to  answer  just  about  any  question  that  could  be  asked 
about  the  data  collected  in  the  system.  Some  training 
is,  however,  required  to  use  these  statistical  packaqes. 

RATING: 
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APPENDIX  B 


INTRODUCTION  TO  SURVEY  FORM  AND  EXPLANATION  OF  RATING  SCALE 


OPERATIONAL  COMPUTER-BASED  TRAINING  SYSTEM  REQUIREMENTS 

A  highly  sophisticated,  prototype  computer-based  training  system  has 
been  developed  and  implemented  at  Lowry  AFB.  This  Advanced  Instructional 
System,  or  AIS,  represents  the  state-of-the-art  in  both  instructional 
and  computer  technology,  and  has  contributed  to  training  tine  savings 
of  30X  or  more  in  a  variety  of  technical  training  areas.  As  a  prototype 
system,  however,  the  AIS  provides  ^unctions  and  capabilities  that  may 
not  be  necessary  in  the  operational  training  environment.  To  realize 
the  full  potential  of  computer-based  systems  for  Air  Force  training,  we 
want  to  find  out  what  functions  or  capabilities  will  have  the  highest 
payoff  for  operational  training. 

You  have  been  selected  to  participate  in  this  survey  to  determine 
what  functions  are  most  valuable  in  your  training  area.  Your  assistance 
in  carefully  analyzing  the  value  of  various  computer-based  functions  is 
extremely  important.  Your  ratings,  along  with  the  ratings  of  the  other 
participants,  will  be  used  in  defining  a  low-cost  computer-managed  and 
computer-assisted  instruction  (CMI/CAI)  system  for  Air  Force  training. 

The  following  pages  describe  the  many  functions  that  a  computer 
could  perform  in  a  training  system.  Please  read  the  description  of  a 
function,  and  then  carefully  consider  what  that  function  could  accomplish 
in  your  parMcular  area  Of  training.  The  question  you  are  to  answer, 
for  each  function,  is: 

"HOW  MUCH  IS  THIS  FUNCTION  WORTH  TO  MY  AREA  OF  AIR  FORCE  TRAINING?" 
Payoff  potential  can  come  from  any  of  the  following: 

0  Reduced  training  time,  students  get  through  faster; 

0  Reduced  administrative  load  on  instructors; 

0  Beliei  use  of  training  resources; 

0  Improved  quality  control  over  students  or  materials; 

0  Improved  manayement/administration  of  training;  and 

0  Improved  materials  developiiient  process,  resulting  in  better  in¬ 
structional  materials  and  reduced  development  time. 

On  the  following  pages,  there  are  eight  brief  general  descriptions 
of  various  areas  in  which  the  computer  could  provide  training  support. 


Under  each  ot  the  general  area  dcsc. 


i  'M *  •: 
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computer-based  training  functions  are  described.  You  are  asked  to  con¬ 
sider  how  much  each  of  these  might  be  worth  to  Air  Force  training  in 
your  particular  area,  and  to  assign  a  rating  of: 


1  -  Very  high  payoff  potential; 

2  =  High  payoff  potential; 

3  =  Moderate  payoff  potential ; 

4  =  Small  payoff  potential; 

5  =  No  payoff  potential;  and 

X  =  No  opinion  -  this  function's  effects  would  be  outside  my  area, 
and  I  cannot  estimate  its  payoff  potential. 

If  there  are  functions  or  features  that  should  be  included  in  a 
computer-based  training  system  for  your  area  of  Air  Force  training,  and 
that  are  not  listed,  please  describe  these  on  the  last  page  of  this 
packet. 

If  you  are  involved  in  resident  training,  then  assume  that  your  area 
of  training  would  be  self-paced  before  the  computer-based  training  system 
is  introduced.  This  is  important,  because  many  of  the  computer  functions 
could  be  quite  valuable  in  self-paced  training  but  of  little  value  in 
conventional  lockstep  instruction.  Remember,  as  you  assign  ratings,  that 
you  are  to  consider  the  payoff  of  each  function  in  self-paced  training. 

If  you  have  any  questions,  please  ask  the  representative  conducting 
this  survey. 
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PREASSESSMENT  DATA  CAN  8E  USED  /  2.57 

RELEVANT  AF  PERSONNEL  DATA  CAN  BE  USED  /  2.53  S  BATCH 
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APPENDIX  D  (CONTINUED) 

FREQUENCIES,  CATEGORIES  PROPORTION 

1  -  5  AND  X  OF  I  &  2 

ITEM  ”1  2  3  4  5  X  MEAN  RATINGS* 
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A  blank  in  this  column  indicates  that  less  than  half  of  the 
participants  rating  an  item  assigned  a  rating  of  1  or  2. 
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APPENDIX  G 


I 


I 


LANGUAGE  FEATURES  REQUIRED  BY  THE  CAI/CMI  FUNCTIONS 

Language  features  were  determined  by  examining  the  necessary 
CAI/CMI  functional  capabilities  from  Appendix  C  and  determining  a 
set  of  suggested  features  for  accomplishing  those  capabilities. 
The  numbers  in  the  "FUNCTIONS"  column  refer  to  the  Appendix  C 
listing  of  CAI  and  CMI  functions.  The  nunbers  in  the  "LANGUAGE 
FEATURES"  column  refer  to  the  listing  of  features  in  Section 
5.2.3  of  this  report. 


FUNCTIONS 


All 


VA-VC2,  VI C3,  VI F 

IA-IVB4,  VIA1-VIC,  VIF, 

VI IA1-VI IA4C,  VIIB6-VI IB7 

IB-186 ,  VA-VIF,  VIIA4C, 
VIID-VIIE7 

IA-IIIC6,  VI IA1-VI IA4B , 
VIIB1-VIIB7 

IB-IB6,  VA-VIIA,  VIIA4C, 
VIIB3-VIIB5,  VIIB8, 
VIID-VIID9,  VIIH1-VIIH3 

VA-VC2,  VIC3-VIE6 

VIC4-VI IA2A,  VIIB1-VIIB8, 
VIIF-VIIH3 

VIE1-VIE4 


LANGUAGE  FEATURES _ 

1,  2,  4,  5,  6,  7,  8,  10,  11, 
12,  13,  14,  15,  17,  20,  21, 
22 

3 

9 

16 

18 

19,  23 

24 

25 

26 


(Pages  124,  125,  and  126  intentionally  left  blank.) 
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APPENDIX  H 


FUNCTIONAL  SPECIFICATION 
FOR  THE 

LOU  COST  CAI/CMI  INSTRUCTIONAL  SYSTEM 

Prepared  Under  Contract 
F33615-78-C-0031 

FOR 

TECHNICAL  TRAINING  DIVISION 
AIR  FORCE  HUMAN  RESOURCES  LABORATORY 
AIR  FORCE  SYSTEMS  COMMAND 
LOWRY  AIR  FORCE  BASE,  COLORADO 

BY 
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This  specification  establishes  th  ;  requirements  for  an  operational ly 
configured,  low  cost,  computer-managed  instruction  (CMI  )/cei;iputer- 
assistoJ  instruction  (CAI)  system.  The  specification  includes  those  CM  1 
and  CAI  functions  identified  as  having  potential  payoff  from  an  oper¬ 
ational  viewpoint,  and  excludes  "nice  to  have"  or  "rich"  capabilities  for 
support  of  research  and  development  (RAD)  and  other  non-operational 
functions . 

The  system  shall  be  a  dedicated  local  system,  rather  than  a  large 
central  system  serving  more  than  one  Technical  Training  Center  (TIC)  or 
a  system  made  up  of  several  stand-alone  mini-systems.  A  modular  approach 
to  expansion  snail  be  incorporated  into  the  design,  to  facilitate  such 
trade-offs  as  fewer  on-line  students  in  exchange  for  heavier  CAI  usage, 
and  to  accommodate  increasing  memory  or  mass  storage  in  order  to  handle 
increased  loads. 

The  system,  including  the  on-line  CM1/CAI  data  base,  shall  be  sired 
to  provide  operational  CMI/CA1  support  for  at  least  five  hundred  (b00) 
students  per  shift  for  two  shifts.  Further  design  assumptions  are: 
these  students  are  distributed  equally  across  five  (5)  different  courses; 
each  student  averages  five  (b)  CMI  transactions  per  shift;  not  more 
than  10,  of  the  students  will  use  CAI  at  any  one  time;  up  to  nine  (9) 
admi nistrators  (instiuctors  or  supervisory  personnel)  can  be  on-line 
simultaneously;  and  batch  processing  is  accomplished  oif-shi ft  (i.e., 
during  periods  of  low  or  no  student  CAI /CMI  load),  except  for  essential 
recovery  of  lessons  which  are  stored  off-line.  The  hardware  system  shall 
be  adequate  to  provide  the  functions  described  in  Section  3.0  of  this 
specification  while  meeting  the  constraints  and  assumption*  described 
above. 


2.0  APPLICABLE  DOCUMENTS 


The  following  documents,  of  the  issue  in  effect  on  the  date  of 
invitation  for  bids  or  request  for  proposals,  form  a  part  of  this  specifi¬ 
cation  to  the  extent  specified  herein.  Any  reference  to  the  following 
documents,  by  basic  number  only,  in  other  paragraphs  herein  shall  be 
deemed  to  apply  to  the  issue  current  at  the  time  of  issuance.  In  the 
event  of  conflict  between  the  documents  referenced  here  and  other  detail 
content  of  Sections  3.0  and  4.0,  the  detailed  requirements  of  Sections 
3.0  and  4.0  shall  be  applicable. 

2.1  GOVERNMENT  DOCUMENTS 

MILITARY  SPECIFICATIONS 

MIL-H-468’j5  Human  Engineering  Requirements  For  Military  Systems, 
Equipment  and  Facilities 

MIL-L-6051D  Electromagnetic  Compatibility  Requirements,  Systems 

MIL-Q-9585A  Quality  Program  Requirements 

MIL-T-23991  Training  Devices,  Military;  General  Specification 
For 

MILITARY  STANDARDS 

MIL-STU-785A  Reliability  Program  For  Systems  and  equipment 
Development  and  Production 

MIL-STD-130  Identification  Marking  of  U.S.  Military  Property 

MIL-STD-14.1B  Standards  and  Specifications,  Order  of  Precedence 
For  The  Selection  of 

MlL-STD-470  Maintainability  Program  Requirements  (For  Systems 
and  Equipments) 

MIL-STD-490  Specification  Practices 

MIL-STD-882  System  Safety  Program  for  Systems  and  Associated 
Equipment:  Requirements  for 

MIL-STD-I472A  Human  Engineering  Design  Criteria  for  Military 
Systems,  Equipment,  and  Facilities 
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3.0  FUNCTIONAL  REQUIREMENTS 


This  Section  describes  the  functional  requirements  for  the  low  cost 
CMI/CAI  system. 

3.1  STUDENT  TESTING  AND  EVALUATION 

The  system  shall  read  students'  test  forms,  score  tests,  provide 
each  student  with  feedback  on  test  results,  and  record  the  results  in  an 
on-line  data  base.  The  test  results  shall  be  made  available  in  printed 
reports  for  evaluation  and  quality  control  purposes.  The  system  shall 
support  at  least  the  following  types  of  tests:  aptitude  tests,  lesson 
and  block  tests,  performance  checks,  and  attitude  measures.  The  types 
of  allowable  questions  shall  include  at  least  true-false,  multiple 
choice,  and  scalar. 

The  system  shall  include  on-line  and  off-line  testing.  The  on-line 
testing  shall  be  supported  by  interactive  terminals.  Off-line  testing 
shall  be  supported  by  management  devices,  which  shall  include  optical 
mark  readers,  badge  readers,  off-line  response  devices,  or  other 
appropriate  devices  to  acquire  input  from  students  and  instructors; 
printers  to  produce  hard  copies  of  student  status  reports*;  and  other 
devices  as  appropriate. 

3.1.1  Test  Scoring  and  Feedback  Capabilities 

The  system  shall  score  those  tests  for  which  test  keys  have 
been  entered  into  the  data  base  (see  Section  3.6).  The  results  from  tests 
shall  be  available  in  printed  format  for  both  the  student  and  instructor. 

Tf  the  test  is  a  pass/fail  test,  training  managers  shall  be  able  to  specify 
the  pass/fail  criteria.  The  criteria  shall  include  at  least  the  following: 

0  Answers  can  be  scored  either  right  or  wrong,  and  a  designated 
percent  correct  can  be  required  to  pass  the  test. 

0  Objectives  can  be  defined  within  the  test  and  a  subset  of  the 
test  questions  can  be  identified  as  relating  to  each  specified 
objective.  It  shall  be  possible  to  use  a  question  for  more  than 
one  objective. 

0  Any  objective  can  be  designated  as  a  critical  objective  and  the 
test  will  be  failed  if  the  objective  is  failed. 

0  A  subset  of  objectives  can  be  defined,  with  a  minimum  number  of 
objectives  from  the  subset  required  to  be  passed  in  order  to  pass 

*  A  status  report  shall  be  printed  as  part  of  each  student  CMI  trans¬ 
action.  Each  status  report  shall  include  appropriate  feedback 
to  the  student  on  the  lesson  or  other  assignment  just  completed, 
and  shall  include  appropriate  direction  as  to  the  student's  next 
action--lesson  assignment,  test  assignment,  etc. 
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the  test.  For  example,  five  objectives  on  a  ten-objective  test 
can  be  specified  as  a  subset,  and  the  system  can  require  that 
three  or  more  of  these  objectives  must  be  passed  or  the  test  will 
be  failed. 

Test  keys  and  other  information  required  to  score  tests  shall  be 
entered  at  an  interactive  terminal. 

The  system  shall  score  performance  checklists  completed  by  instruc¬ 
tors.  Instructors  shall  be  able  to  certify  that  a  student  has  either 
passed  or  failed  any  of  the  tests,  including  performance  checks,  from  an 
interactive  terminal  or  by  entering  appropriate  information  via  a  manage¬ 
ment  device. 

3.1.2  On-Line  Testing  Capabilities 

The  system  shall  provide  the  capability  for  on-line  testing 
via  an  interactive  computer  terminal.  For  on-line  testing,  the  order*; of 
guestions  and  the  sequence  of  alternate  answers  for  each  question  can  he 
scrambled  automatically  for  each  test  administration.  The  order  of  the 
alternatives  shall  be  either  fixed  or  random  as  specified  h>  the  author*. 
The  author  shall  be  able  to  make  the  extent  of  testing  dependent  on  the 
student's  responses  to  earlier  questions  in  the  test.  A  hard  copy  of  the 
results  of  on-line  tests  shall  be  available  from  a  management  device  on 
request.  Responses  to  on-line  student  test  inputs  shall  be  in  a  real 
time  mode.  The  time  lapse  from  transmission  of  student  input  to  start 
of  displaying  feedback  shall  not  exceed  throe  (3)  seconds  in  more  than 
10*  of  the  cases.  The  time  lapse  from  beginning  to  completion  of  the 
response  shall  not  average  greater  than  twelve  (12)  seconds. 

3.1.3  Determination  of  Pre-Course  Student  Characteristics 

The  system  shall  provide  for  storage  and  retrieval  of  student 
characteristics  data  collected  before  or  at  the  beginning  of  training.  It 
shall  be  possible  to  use  the  student  characteristics  data  in  examining 
eliminations  as  a  function  of  aptitude  and  of  years  and  types  of  pre¬ 
service  schooling,  and  in  predicting  individual  rates  of  progress  toward 
course  completion.  The  individual  characteristics  i nformation  shall  be 
stored  in  the  on-line  CMI  data  base  and  shall  be  available  to  the  CHI 
system  as  long  as  the  individual  is  enrolled  in  the  course. 

The  system  shall  be  capable  of  storing  and  retrieving  the  following 
pre-course  data  for  all  students  enrolled  in  courses  operating  with  this 
system; 

*NOTF:  Within  the  context  of  this  specification,  "author"  is  used 
as  a  generic  term  to  describe  Air  Force  personnel --super¬ 
visors,  managers,  administrators,  instructors--involved  with 
operating  this  system  and,  in  particular,  with  entering  data 
into  the  system. 
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Relevant  Air  Force  personnel  data  (e.g.,  ASVAR  or  ATQT  scores, 
and  such  biographical  data  as  previous  service  training  and  duty 
assi  gnrients ) ; 

0  knowledge  of  che  course  material  based  on  criterion  referenced 
pre-tests  or  other  pre-course  measures  of  knowledge/proficiency 
on  training  objectives; 

0  Learning  process  skills  (ability,  attitude,  or  interest  measures 
such  as  reading  comprehension  and  study  habits);  and 

0  Course-specific  critical  entry  skills  (aptitudes,  abilities,  and 
interests  that  are  specifically  related  to  success  in  a  particular 
training  specialty,  such  as  mathematical  ability  or  interest  in 
the  career  field). 

3.1.4  Determination  of  Hi  thi  n-C.ourse  Student  Attitudes  and 

Interests 

The  system  shall  make  precision  for  tests  to  measure  the 
attitudes  and  interests  of  students.  Pro-course,  wi thin-course,  and  post¬ 
course  tests  of  attitudes  and  interests  shall  bo  accommodated.  The 
results  cf  the  tests  shall  be  stored  in  the  on-line  CM I  data  base  for 
use  in  determining  training  assignments,  evaluating  instructional 
materials  and  procedures,  predicting  completion  limes  and  rate?  ot  pro¬ 
gress,  and  predicting  performance ,  as  specified  by  training  managers. 

3.2  STUDENT  ASSIGNMENT 

The  system  shall  assign  students  individually  to  lessons,  tests,  and 
other  instructional  al ternati ves .  The  system  shall  be  structured  to 
allow  the  training  manager  to  specify  what  combination  of  data  on  lesson 
characteristics,  resource  avai labi 1 i ty ,  student  characteristics,  and 
student  performance  will  be  used  in  determining  the  assignments  of 
individual  students  to  training  al ternati ves. 

A  student  shall  receive  the  next  individual  assignment  immedi ate ly 
after  completing  the  preceding  lesson.  Training  resources  shall  be 
scheduled  by  the  computer  system  to  attempt  to  maximize  throughput.  The 
computer  shall  track  student  progress  through  the  lessons  and  alternatives 
in  the  various  course  versions.  The  system  shall  provide  reports  that  can, 
as  desired,  replace  manual  tracking  of  student  progress  by  instructors. 

3.2.1  Assignment  to  Alternative  Course  Versions 

The  system  shall  support  course  versions  which  differ  as  to 
content  (e.g.,  for  different  shredouts  of  a  specialty,  or  for  testing  and 
implementing  new  training  materials).  A  student  enrolled  in  a  particular 
course  version  shall  be  assigned  only  the  training  materials  fo»‘  that 
version.  The  form  or  procedure  used  to  register  a  student  in  i  course 
shall  provide  for  identifying  the  student's  course  and  version. 


3.2.2  Determination  of  the  Order  of  Assignments 

The  system  shall  have  the  capability  to  determine  and  assign 
the  next  lesson  alternative  for  an  individual  student  in  all  modes 
except  block  remediation.  The  instructor  shall  not  be  required  to 
manually  track  student  progress,  or  to  determine  assignments  to  train¬ 
ing  alternatives,  and  shall  have  capability  to  perform  manual  override 
of  any  assignment.  Based  on  data  in  the  CMI/CAI  data  base  and  subject 
to  options  selected  by  authors  or  data  base  managers,  the  system  shall 
be  capable  of  using  at  least  the  following  information  in  determining 
a  trainee's  next  assignment: 

O 

The  assignments  completed  by  the  trainee  to  date; 

The  performance  of  the  trainee  on  previous  assignments  (times  and 
test  results  on  previous  assignments); 

The  availability  of  the  resources  required  for  existing  and 
allowable  lesson  alternatives;  and 

The  current  distribution  of  students,  the  numbers  of  students 
required  for  any  team  tasks,  and  how  best  to  assemble  teams  of 
trainees  for  team  tasks. 

3.2.3  Assignment  to  Alternatives  Within  lessons 

The  system  shall  make  individual  student  assignments  to 
alternatives  within  a  lesson.  These  can  include  alternate  media  (e.g., 
printed  vs.  audio-visual)  or  different  instructional  approaches  (e.g., 
easy  vs.  difficult).  The  assignments  can  be  based  on  at  least  the 
following  considerations: 

0  Availability  of  the  particular  media  or  lesson; 

0  Prior  test  scores,  times-to-complete,  or  other  wi thi n-course  data; 

0  The  student's  pre-course  characteristics  (e.g.,  general  aptitudes, 
abilities,  attitudes,  interests,  or  other  pre-course  data); 

°  Simple  "if.. .then"  logical  rules  which  may  be  used  to  select 
alternatives;  and 

0  Predicted  student  performance. 

Student  characteristics  and  past  performance  can  be  used  to  predict 
student  performance.  The  student  shall  be  assiqned  to  the  media  or 
lesson  predicted  to  result  in  the  best  student  performance.  The  system 
shall  accommodate  at  least  two  assignment  rules  for  each  iesson--best 
predicted  time  and  best  predicted  score--which  can  be  specified  at  the 
lesson  level  by  an  author.  If  the  score  rule  is  selected,  the  student 
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shall  be  assigned  to  the  treatment  with  the  highest  predicted  score. 

If  the  time  rule  is  used,  the  system  shall  assign  the  student  to  the 
treatment  with  the  shortest  predicted  time.  If  the  time  rule  is 
specified  and  the  student  is  predicted  to  fail  all  of  the  treatments, 
the  system  will  use  the  score  rule.  These  rules  will  be  used  only  for 
a  student's  first  attempt  on  a  lesson.  Subsequent  assignments  shall 
be  made  using  remediation  rules  (Section  3.2.4).  Additionally,  resource 
availability  and  system  throughput  constraints  shall  override  the  time 
and  the  score  rules. 

The  system  shall  support  random  assignment  of  a  specifiable  per¬ 
centage  of  the  students  to  the  treatments  within  a  lesson.  If  15:; 
random  assignment  is  specified,  then  a  randomly  selected  15?;  of  the 
students  shall  be  assigned  randomly  to  the  available  instructional 
alternatives  for  that  lesson.  The  random  assignment  function  will  be 
used  in  collecting  data  to  establish  assignment  rules  and  to  verify 
the  effectiveness  of  such  rules. 

3.2.4  Assignment  to  Alternative  Remediation  Activities 

The  system  shall  make  assignments  to  the  available  alternate 
remediation  materials  and  media.  At  least  three  categories  of 
remediation  shall  be  supported: 

0  Lesson  Remediation  -  The  system  shall  support  alternatives  for 
use  in  remediation  only.  The  system  shall  assign  a  student  to 
the  "best"  available  remediation  alternative  for  second  and  sub¬ 
sequent  lesson  attempts,  with  "best"  defined  to  be  the  one 
alternative  chosen  from  available  alternatives  based  on  the  con¬ 
siderations  listed  in  Section  3.2.3. 

0  Lesson  Group  Remediation  -  The  system  shall  assign  the  best 
available  remediation  alternatives  within  a  related  group  of 
lessons. 

0  Block  or  Course  Level  Remediation  -  The  system  shall  assign  the 
best  available  remediation  alternatives  for  the  objectives  that 
the  student  is  having  difficulty  mastering  within  the  block  or 
course. 

The  alternate  remedial  selections  shall  be  accompl ished  by  the 
computer.  It  shall  be  possible  for  instructors  to  override  system 
decisions  via  management  devices  and  from  interactive  terminals. 

3.2.5  Test  Selection  and  Assignment 

The  system  shall  provide  the  capability  of  assigning  students 
to  tests.  For  each  student  at  each  testing  point,  the  System  shall 
assign  an  on-line  (CAI)  or  an  off-line  (paper  and  pencil  or  performance) 
test,  as  available  and  appropriate.  If  more  than  one  version  of  a  test 
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Is  available,  the  alternates  shall  be  assigned  randomly.  Any  versions 
already  attempted  by  the  student  shall  be  excluded  on  subsequent 
attempts.  Capability  shall  be  provided  for  manual  override  of  any  test 
assignment  by  an  instructor.  On-line  programs  shall  provide  the 
necessary  data  entry  capabilities  for  implementing  this  function. 

3.2.6  Assignment  to  Supplementary  Skill  Training  at  Course  Entry 

The  system  shall  be  capable  of  identifying  those  students 
with  deficiencies  in  critical  entry  skills  or  in  learning  skill  areas 
and  of  assigning  students  to  supplementary  or  remedial  skill  training. 

The  system  shall  identify  such  students  based  on  critiera  established 
by  course  managers  for  performance  on  aptitude,  ability,  interest,  or 
other  precourse  or  within  course  measures.  Supplementary/remedial  train¬ 
ing  lessons  shall  subsequently  be  assigned  to  students  in  accordance 
with  hierarchy  and  resource  availability  constraints  established  in  the 
CMI/CAI  data  base  by  course  managers. 

3.2.7  Additional  Resource  Management  Considerations 

Based  on  information  entered  into  the  CA1/CM1  data  base,  the 
system  shall  track  which  resources  are  required  for  each  lesson,  if  they 
ate  available,  and  where  they  are  located.  Managed  resources  can  include 
classroom  spaces,  lesson  materials,  media  devices,  work  stations, 
simulators,  and  other  training  equipment.  The  system  shall  be  able  to 
assign  and  direct  students  to  training  resources  outside  of  their  assigned 
learning  centers,  to  allow  effective  sharing  of  resources  by  students 
in  several  learning  centers. 

The  system  shall  assign  students  in  a  manner  that  will  (1)  tend  to 
distribute  students  evenly  throughout  the  course,  (2)  attempt  to  keep 
as  many  assignment  options  as  possible  open  for  the  students,  and  (3) 
make  efficient  use  of  available  resources.  Students  shall  be  assigned 
to  the  learning  centers  with  the  hiqhest  percentages  of  available  space, 
and  shall  be  assigned  to  blocks  in  such  a  manner  as  to  optimize  the 
distribution  of  students  across  learning  centers.  In  the  process,  the 
system  shall  consider  team  assembly,  desired  loadings  within  blocks,  and 
the  capacities  of  learning  centers.  These  considerations  shall  not, 
however,  override  the  hierarch, i cal  constraints  or  priority  given  to 
each  block. 

The  computer  system  shall  base  the  above  management  decisions  on  data 
entered  into  the  on-line  data  base.  Once  a  course  version  is  operative, 
training  management  personnel  shall  be  required  to  enter  only  course 
changes  as  they  occur.  These  include  changes  in  the  course  structure 
(e.g.,  lessons  added  or  removed,  or  the  prerequisite  order  of  assign¬ 
ment  changed),  and  resource  availability  (e.g.,  new  equipment  added, 
and  equipment  removed,  inoperative  or  in  naintenuiKe) .  The  changes  shall 
be  entered  using  on-line  programs  and  shall  take  effect  when  entered. 


3.3  S7U3ENT  PitOGPEcS  MANAGEMENT 

The  system  snail  suuport  management  of  a  trainee's  progress  through 
a  self-paced  individualized  training  course.  These  capabilities  shall 
range  from  lesson  scheduling  to  progress  prediction  and  management. 

Both  the  student  and  the  instructor  can  he  informed,  at  any  time,  as  to 
the  student's  progress --ahead  of,  behind,  or  on  schedule. 

3.3.1  Scheduling  Student  Entries  Into  The  Course 

At  the  end  of  each  training  ua.y,  the  system  shall  identify 
the  number  of  vacancies  in  each  course  operating  under  the  system.  This 
information  will  allow  the  student  squadron  or  other  administrative 
authority  to  effectively  plan  and  program  course  entries  and  minimize 
student  delay  awaiting  training. 

3.3.2  Graduation  Forecasting 

The  system  shall,  as  pare  of  the  student  monitoring  and 
manager, .ont  process  (Section  3.3.3),  predict  the  date  on  which  a  student 
will  complete  the  course.  Learning  center  rosters  shall  be  available 
on  request,  and  shall  identify  students  and  their  predicted  graduation 
dates  during  the  date/time  interval  specified  in  the  training  manager's 
request.  The  rosters  shall  be  available  in  printed  form,  and  for  display 
at  an  interactive  terminal.  The  average  response  time,  from  submission 
of  a  request  to  completing  printout  of  a  roster,  shall  not  exceed  two 
(2)  seconds  per  student.  The  system  shall  also,  on  request,  generate 
graduation  reports  for  the  local  training  center  personnel  office. 

These  reports  shall  be  generated  on  the  central  site  line  printer. 

The  system  shall  be  configured  to  facilitate  interface  directly  with 
the  training  center  personnel  management  computer  system.  To  implement 
this  function,  the  Air  Force  will  specify  the  format  of  the  data  and 
the  medium  in  which  the  training  center  personnel  computer  will  receive 
the  required  information.  Any  required  changes  to  the  training  center 
personnel  computer  software  will  be  accomplished  by  Air  Force  personnel. 

3.3.3  Monitoring  and  Management  of  Student  Progress  to  a  Target. 

The  system  shall  determine  a  target  progress  rate  for  each 
student.  The  rate  for  a  student  shall  be  used  to  generate  a  targeted 
time  for  completion  of  the  course  and  for  completion  of  individual 
ulocks.  The  target  progress  rate  for  a  student  shall  be  based  on  pre- 
ccurse  data  „nd  on  available  wi thi n-course  performance  data  The  targets 
shall  te  relative  to  average  completion  times  for  blocks  and  course.  A 
student's  target  completion  date  shall  be  calculated  based  on  the  target 
progress  rate,  expected  shift  length,  predicted  days  the  student  will 
be  in  class,  the  student's  'erformance  record,  and  the  percentage  of  the 
course  remaining.  Each  student  shall  have  access  to  his/her  target  time 
to  complete  for  each  block.  Both  the  student  and  the  instructor  shall 
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receive  feedback  on  the  student's  actual  progress  rate  relative  to 
targeted  progress  rate.  The  student  shall  be  given  information,  on 
the  first  status  report  of  each  training  day,  that  will  let  the  stuaent 
determine  how  actual  progress  to  date  compares  with  predicted  progress. 

3.4  SUPPORT  FOR  GUIDANCE  AND  COUNSELING 

The  system  shall  provide  computer  generated  reports  to  support 
student  guidance  and  counseling  by  instructors  and  course  management 
personnel.  If  the  necessary  tests  are  included  in  the  pre-course 
assessment,  the  reports  shall  identify  entry  skill  deficiencies  as  well 
as  deficiencies  in  wi thin-course  performance.  The  instructors  will 
determine  which  students  are  to  receive  counseling,  and  can  retrieve 
from  the  computer  both  the  background  characteristics  and  wi thin-course 
performance  information  needed  to  support  their  guidance  and  counseling 
activities. 

3.4.1  Identification  of  Students  Needing  Special  Attention  at 

Course  Entry 

The  system  shall  identify  students  predicted  to  be  proficient 
or  deficient  with  respect  to  mastery  of  course  or  training  objectives 
specified  by  course  personnel.  This  identification  and  reporting  is 
Initiated  at  the  beginning  of  the  training  process  to  focus  instructor/ 
management  attention  toward  students  most  in  need  of  special  training 
procedures,  guidance,  and  counseling. 

The  identification  process  shall  produce  computer  generated  lists 
of  those  students  measured  as  being  either  proficient  or  deficient  on 
the  basis  of  their  pre-course  assessments.  This  initial  listing  shall 
report,  by  student,  individual  scores  on  the  pre-course  critical  entry 
skill  variables  (e.g.,  reading,  math,  or  study  skills).  A  second 
report  shall  provide  individual  scores  for  all  students  on  those 
variables  identified  as  critical  entry  skills,  and  shall  also  flag 
deficient  students  and  their  related  scores.  These  reports  shall  be 
available  for  display  at  the  interactive  terminals. 

3.4.2  Identification  of  Marginal  Student  Performance 

The  system  shall  produce  reports  identifying  and  reporting 
students  predicted  to  have  difficulty  mastering  further  training 
objectives  based  on  recorded  wi thin-course  performance.  Tin;  reports 
shall  be  available  on  demand  and  utilized  to  direct  timely  remedial/ 
counseling  assistance  to  students  most  in  need.  The  system  shall  be 
capable  of  assigning  those  students  identi  fietf  as  marginal  to 
specialized  remedial  training  (e.g.,  to  special  instructor  tutorials, 
remediation  sessions,  or  special  skill  training  for  correcting  learning 
or  study  skill  weaknesses). 

The  system-generated  reports  shall  include  a  list  of  these  students 


not  meeting  predetermined  limits  of  satisfactory  predicted  progress 
rates,  or  scoring  below  the  training  manager's  criterion  levels  for 
training  objectives.  They  shall  include  actual  time  ahd  score  data. 

The  students  below  the  course-established  mini  mums  for  acceptable  per¬ 
formance  will  be  flagged  as  possible  candidates  for  elimination  or 
other  established  administrative  action. 

3.5  COMPUTER-AIDED  INSTRUCTION  (CAI) 

Lesson  materials  can  be  entered  into,  stored  in,  and  retrieved  and 
delivered  by  the  computer  system.  When  assigned  by  the  system  and 
requested  by  the  student,  they  can  be  displayed  at  an  interactive  terminal. 
The  lesson  materials  can  include  text,  questions,  graphics,  and  special 
purpose  programs.  Student  interactions  can  range  from  a  simple  "turning 
the  pages"  type  of  program  through  very  sophisticated  interactive 
exchanges  via  special  purpose  programs.  Student  interactions  are 
principally  via  a  terminal  keyboard  and,  if  appropriate,  via  a  touch 
panel,  light  pen,  or  other  device. 

3.5.1  CAI  Applications 

The  CAI  provided  by  this  system  shall  support  at  least  the 
following  five  CAI  functions  for  use  in  Air  Force  resident  technical 
training: 

0  Drill  and  Practice,  with  correction  and  guidance,  of  basic  skills 
and  knowledges  learned  via  other  media; 

0  Tutorial--use  of  CAI  for  lessons  which  are  particularly  difficult; 

0  Simulat1on--use  of  soeclal  purpose  programs  to  simulate  equipment 
operation  or  maintenance  processes; 

0  Review  and  Remediation--use  ot  CAI  to  quickly  review  the  content 
of  a  block  prior  to  a  block  test  or  to  remediate  failed  objectives 
following  a  block  test  failure;  and 

0  Study  Skills--use  of  CAI  to  teach  specific  study  skills  to  students 
who  are  deficient  in  these  areas. 

3.5.2  CAI  Capabilities 

A  CAI  lesson  shall  consist  of  a  series  of  frames,  or  equiv¬ 
alent  means  of  storing  and  presenting  information.  The  system  s ha  1 1 
support  at  least  the  following  types  of  frames,  or  their  equivalents: 
text  frames  that  can  contain  at  least  4  pages  of  information  (a  page  can 
contain  at  least  21  lines);  question  frames;  and  documentation  frames 
which  the  student  cannot  access.  The  documentation  frames  are  reserved 
for  course  authors  to  record  development  status  data.  The  system  snail 
provide  for  at  least  100  objectives  per  lesson  and  100  frames  per 


objective. 

The  system  shall  support  CAT  lessons  of  varying  levels  of  sophis¬ 
tication.  The  basic  CA1  lesson  will  consist  of  text  presentation,  and 
will  allow  response  to  multiple-choice  and  true-false  questions  via  a 
typewriter-1  ike  keyboard.  The  system  shall  be  able  to  provide  feedback 
messages  specific  to  the  Individual  student's  responses  and  number  of 
attempts  on  the  current  objective.  The  system  shall  also  allow  branching 
to  any  available  frame  addressing  the  current  objective,  or  to  the  first 
frame  of  any  other  objective  within  the  lesson.  The  lesson  progression 
shall  be  adaptable  to  the  student's  responses.  The  branching  selection 
shall  allow  branching  to  any  available  frame  addressing  the  current 
objective  or  to  the  first  frame  of  any  other  objective  within  the  lesson. 
The  lesson  progression  shall  be  adaptable  to  the  student's  responses. 

The  branching  selection  shall  be  based  on  the  student's  prior  responses/ 
performance  within  the  lesson. 

For  more  sophisticated  CAI  lessons,  the  system  shall  support 
graphics  presentations,  color  displays,  student  entry  of  constructed 
responses  (open-ended  English  answers),  control  of  supplementary  slide 
or  filmstrip  presentations,  control  of  audio  tapes,  and  adaptations 
based  on  the  CMI  on-line  data  base  (e.q.,  ASVAB  or  AFQT  scores,  pre¬ 
course  assessments,  and  wi thi n-course  performance  data).  The  system 
shall  support  the  audio  tape  player  and  the  projector  via  an  external 
computer-controlled  jack  on  the  interactive  terminal.  To  facilitate 
student  interactions  in  these  CAI  lessons,  the  system  shall  support  the 
usual  keyboard  inputs  and  shall  also  support  an  additional  device  such 
as  a  light  pen  or  touch  panel.  To  facilitate  preparation  of  graphic 
displays,  the  system  shall  support  a  digitizing  tablet  or  equivalent 
device. 

The  system  shall  have  the  capacity  for  on-line  mass  storage  of  a 
minimum  of  lbO  lessons,  available  for  immediate  access.  The  lessons 
are  assumed  to  average  three  objectives  per  lesson,  60  frames  (or 
equivalent)  per  objective,  2  pages  per  frame,  and  lb  lines  per  page 
(average  line  length,  60  characters).  Lessons  in  daily  use  shall  he 
available  on  the  system  at  all  times,  with  no  more  than  a  ten  (10)  second 
delay  in  initiating  student  access.  When  a  student  has  indicated  com¬ 
pletion  of  a  frame,  the  average  time  for  the  system  to  determine  and 
initiate  the  next  display  shall  not  exceed  three  (3)  seconds.  The 
average  time  to  complete  the  display  shall  not  exceed  twelve  (1?) 
seconds. 

Inactive  lessons  which  are  stored  off-line  shall  he  recoverable  for 
normal  on-line  accessibility  not  later  than  the  start  of  the  next  training 
day  following  the  reguest  tor  restoration. 

3 . b . 3  On-line  Learning  Proccss/Special i zed  Skill  Trainimi 

The  CAI  system  shall  support  presentation,  to  either  students 
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nr  instructors,  of  specialized  training  in  interactive  formats.  This 
trainin’)  mav  include,  for  example,  materials  designed  to  teach  study 
habits  and  skills,  test  takinq  skills,  and  memorization  skills  to 
students,  or  to  teach  diagnostic,  tutorial,  authoring  or  counseling 
skills  to  i ns tructors . 

3.6  CM I  DATA  BASE  AND  CAI  MATERIALS  PRODUCTION  AND  MAINTENANCE 

To  effectively  perform  CA1/CMI  traininn  functions,  the  system 
must  he  informed  as  to  the  trainees  (name,  social  security  number, 
course,  etc.),  the  course  structure  (tests,  lessons,  resources,  allow¬ 
able  sequences  of  lessons,  etc.),  and  student  performance  (lessons 
completed,  test  scores  within  the  current  block,  block  completion  times, 
etc.).  In  the  case  of  CAI,  all  of  the  lesson  information  (text, 
questions,  graphics,  branching  instructions,  etc.)  must  be  stored  with¬ 
in  the  computer  system.  The  following  paragraphs  describe  the 
requirements  for  entering,  storing,  and  maintaining  these  types  of 
in  fori  lalion. 

3.6.1  CHI  Course  Definition  Data  Base 

The  system  shall  provide  the  capabilities  which  will  allow 
training  managers  to  enter  the  course  structure  into  the  system.  The 
program(s)  shall  run  on- 1 ino-- i .e. ,  a  trainin’)  manager  shall  he  able  to 
use  an  interactive  terminal  to  directly  enter,  store,  display,  and  change 
course  related  information.  To  facilitate  use,  the  on-line  system  shall 
use  graphics  to  display  the  course  structure  (i.e,,  the  allowable  paths 
through  the  lesson  materials),  and  there  shall  he  an  on-line  capability 
to  access  each  file  in  the  CA I /CM I  data  base.  Additionally,  it  shall  not 
be  necessary  for  authors  to  be  familiar  with  any  programming  language 
in  order  to  use  these  capabilities  effectively. 

3.6.2  Automatic  Validation  of  Course  Data  Base  Integrity 

Uefore  a  new  course  or  course  version  is  implemented, 
necessary  information/description  relative  to  all  of  the  blocks,  lessons, 
courseware,  and  tests  in  the  course  must  be  entered  into  the  data  base. 
The  system  shall  include  one  or  more  computer  programs  for  determining 
if  all  required  data  base  records  defining  the  course  are  present  and 
consistent.  The  program(s)  shall  be  designed  to  facilitate  use  by 
authors  and  shall  not  require  knowledge  of  a  programming  language.  To 
the  greatest  extent  possible,  the  program(s)  shall  update  the  on-line 
data  base,  if  this  capability  Is  enabled  by  the  training  manager.  The 
program(s)  shall  run  in  not  more  than  fifteen  (16)  minutes  in  validating 
the  data  base  for  a  course. 

3.6.3  Production  of  On-Line  Tests 

The  CAI  system  shall  include  the  capability  for  on-line  test¬ 
ing.  This  shall  he  implemented  via  an  on-line  test  item  editor,  or 
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equivalent  capability,  for  entering  test  questions,  their  alternatives, 
and  the  necessary  control  information.  This  capability  shall  allow  the 
author  to  enter  test  items  by  typing  text  at  an  interactive  terminal, 
and  shall  also  format  the  questions  fcr  the  author  and  allow  the  author 
to  decide  whether  or  not  to  (a)  randomize  the  presentation  order  of 
the  test  items  and  the  alternatives  within  test  items,  (b)  control  the 
number  of  attempts  the  student  can  make  on  each  item,  and  (c)  enter 
student  feedback  messages  to  follow  either  correct  or  incorrect  answers. 

3.6.4  CA1  Authoring 

The  system  shall  provide  an  on-line  CAI  authoring  editor,  or 
equivalent  capability,  which  shall  assist  in  structuring  the  authoring 
tasks.  This  capability  shall  provide  standardized  formats  for  text  and 
question  frames,  and  shall  provide  the  capability  to  copy  or  share  text 
frames,  questions,  and  graphics  between  lessons.  A  users  manual  shall 
be  provided  for  the  CAI  production  process.  Additional  instruction  shall 
be  displayable  during  the  authoring  process  via  a  HELP  reouest.  The 
HELP  information  shall  include  operating  instructions  for  the  program, 
plus  guidelines  for  preferred  instructional  practices.  This  capability 
shall  be  simple  enough  that  an  author  without  prior  programming 
experience  can  learn  to  use  it  in  not  more  than  two  weeks.  An  author 
shall  have  access  to  any  frame  in  any  lesson  that  is  allowable  within 
system  security  constraints.  Students  shall  not  dp  able  to  access 
lessons  in  the  author  mode.  Additionally,  the  program(s)  shall  provide 
capability  to  readily  change  any  of  the  course  materials  *. lored  in  the 
on-line  data  base. 

3.6.6  Production  of  Off-Line  Materials 

It  shall  be  possible  to  use  the  system  in  producing  tests 
and  instructional  materials  which  are  intended  for  use  off-line  (e.g., 
programmed  texts).  Programs  shall  be  designed  for  use  in  writing, 
reviewing,  and  revising  the  tests  and  instructional  materials  and  for 
printing  the  final  product  in  a  format  ready  for  reproduction.  It  shall 
be  possible  to  print  a  limited  number  of  copies  in  order  to  facilitate 
early  reviews  and  small  group  tryouts.  This  system  capability  shall 
also  facilitate  conducting  small  qroup  tryouts  on-line,  and  shall  he 
useable  for  acquiring  information  on  the  students'  reactions  to  in¬ 
structional  materials. 

3.6.6.  Production  and  Maintenance  of  Course  Documents 

An  interactive  capability  similar  to  that  provided  for  tost 
items  and  CAI  authoring  shall  be  provided  for  developing,  storing,  main¬ 
taining,  and  retrieving  course  documentation  such  as  Plans  Of  Instruction 
(POls),  course  charts,  and  lesson  outlines.  These  documents  shall  be 
available  at  any  time  for  review  or  revision,  and  limited  lumbers  of 
copies  can  be  printed  as  needed. 
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3.7  INFORMATION  RETRIEVAL  AND  REPORTS 


Information  relative  to  the  trainee.'.,  courses,  course  materials, 
tests,  test  items,  and  other  components  of  the  system  shall  be  stored, 
and  available  for  periodic  (hourly,  daily,  weekly,  monthly...)  analyses 
and  reports.  These  functions  are  intended  principally  to  support  in¬ 
structors,  managers,  and  administrators.  They  can  be  valuable  to  in¬ 
structors  involved  with  course  management  and  student  guidance.  They 
can  also  he  valuable  in  providing  an  effective  quality  control  mechanism 
for  imoroving  the  management  and  administration  of  training,  and 
evaluating  and  improving  the  process  of  training  pronran  development  and 
implementation.  A  report  requested  at  a  terminal  shall  be  provided  to 
the  reouester  only  if  security  requirements  for  access  to  the  requested 
information  are  sc.  ied. 

3.7.1  Reports  for  Instructors 

(lata  shall  be  collected,  stored,  and  made  available  in 
reports  structured  to  meet  instructor  needs.  The  reports  shall  include 
but  not  necessarily  be  limiteJ  to  the  following:  student  rosters, 
individual  block  progress  reports,  individual  biographical  reports,  and 
absence  summaries  for  learning  centers.  They  shall  be  available  via 
either  the  interactive  te-  inals  or  in  printed  form.  The  instructor 
shall  also  have  access  ■  the  terminals  to  all  other  student  data  not 
included  in  the  above  ...me  reports,  but  subject  to  established  security 
constraints . 


3.  7. 1.1  Learning  Center  or  Classroom  Rosters.  The  system 
shall  provide,  on  request,  via  either  an  interactive  terminal  o>*  in 
printed  form,  at  least  four  types  of  rosters:  resource,  assignment,  time 
management.,  and  homework.  The  rosters  shall  include  social  security 
numbers  and  names  for  the  students  included  on  the  list.  The  resource 
roster  will  list  those  students  in  the  learning  center  who  have  resources 
assigned  to  them,  the  types  of  resources,  and  the  learning  centers  to 
which  the  resources  are  assigned.  The  assignment  roster  will  list  the 
current  training  activity  assignment  for  each  student  by  learning  center. 
The  time  management  roster  will  include  carrel  (or  position)  number, 
absence  status,  current  block,  and  rate  of  progress  for  individual  students 
by  learning  center.  The  homework  roster  will  identify  the  amount  of  the 
course  completed  to  date  and  the  proportion  completed  through  voluntary 
homework  for  individual  students  by  learning  center.  After  the  terminal 
has  transmitted  a  roster  request  to  the  central  site  computer,  the 
average  processing  time  per  student  shall  not  exceed  one  (1)  second. 

3.7.1.?  Individual  Block  Progress  Reports.  The  individual 
block  progress  report  will  sirnmarlze  student  progress  for  the  block 
specified.  The  information  in  the  block  progress  report  shall  include 
at  least  the  following: 

0  Lessons  completed  in  the  block; 
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Numbers  of  attempts,  and  scores  on  each  attempt,  for  each  lesson 
and  test; 

0  Objectives  failed  on  each  test;  and 

0  Time  spent  on  each  attempt  and  total  time  spent  In  the  block. 

3. 7. 1.3  Individual  Student  History  Reports.  The  Individual 
student  history  report  will  Include  the  student's  scores  on  pre-course 
aptitude  and  interest  measures,  scores  on  any  critical  entry  skill  tests, 
time  spent  in  remediation  following  a  block  failure,  and  any  lesson 
objectives  that  were  failed  more  than  twice. 

3. 7. 1.4  Absence  Summary.  The  absence  summary  shall  be  a 
block-by-block  summary  of  the  absences  reported  to  the  system  and  shall 
be  capable  of  replacing  manual  absentee  reporting.  It  shall  indicate 
when  the  student  was  reported  absent,  when  the  student  returned,  how 
long  the  absences  lasted,  and  the  reasons  for  the  absences. 

.3.  7. 1.5  On-Line  Display  of  Student  Data.  Capabilities  shall 
be  provided  for  retrieval  and  display  of  available  student  information  in 
response  to  requests  which  meet  security  requirements. 

3.7.2  Training  Management  Reports 

The  following  i nformatir., ,  intended  for  use  by  training 
managers  (shift  supervisors  and  eb,ve),  shall  be  provided  in  the  form 
of  standard  reports  available  on  request  to  authorized  users: 

0  Students  awaiting  training,  by  course; 

0  Numbers  of  students  in  training  by  course,  learning  center,  and 
Technical  Training  Center; 

0  Distributions  of  completion  times  for  individual  blocks; 

0  Distributions  of  course  completion  times; 

0  Differences  in  student  performance  between  parallel  learning 
centers*  or  Technical  Training  Centers; 

0  Instructional  resource  (e.g. ,  facilities  and  training  devices) 
util ization; 

0  Utilization  and  qualifications  of  instructors; 

*  Learning  centers  are  "parallel''  if  the  same  course  content  is 
taught  in  the  centers,  and  are  not  parallel  if  the  centers  teach 
different  content. 
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Summary  reports  of  student  completion  times  and  final  grades,  hy 
block  and  course;  and 

Summary  reports  of  di senrol Iment  and  elimination  data. 

3.7.3  Course  Evaluation  Seminary 

The  system  shall  include  a  Course  Evaluation  Sur.iuary  (CES) 
which  summarizes  student  performance  on  individual  lessons  within  blocks 
and  for  entire  blocks.  It  is  intended  for  use  in  monitoring  and 
evaluating  small-group  tryouts  of  new  materials  as  well  as  student  per¬ 
formance  on  established  materials.  The  CES  shall  contain  at  least  the 
following  items  of  information; 

0  Fi rst-attempt  lesson  and  block  test  socres; 

0  Final  lesson  and  block  test  scores; 


Means  and  standard  deviations  of  first-attempt  lesson  and  block 
test  scores; 

Means  and  standard  deviations  of  final  lesson  and  block  test 
scores; 

Means  and  standard  deviations  of  first-attempt  lesson  and  block 
times ; 

Means  and  standard  deviations  of  lesson  and  block  completion 
times ; 

Separate  score  and  time  data  for  alternative  modules  within 
lessons ; 

Score  data  by  objective  within  lessons; 

First-attempt  lesson  failure  rates; 

First- at  tempt  objective  failure  rates; 


o 


Correlations  indicating  the  relationship  between  predicted  and 
actual  lesson  and  block  times  and  scores;  and 


0  Summarized  data  on  lessons,  modules  and  objectives  with  first- 
attempt  failure  rates  above  a  specifiable  percentage. 


The  CES  shall  be  a  batch  mode  report.  An  on-line  program  shall  be 
provided,  however,  for  submitting  requests  for  the  CES  for  off-shift 
process! ng. 
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3.7.4  Test  Item  Evaluation  Report 

The  system  shall  provide  a  Test  Item  Evaluation  (TIE)  report 
that  provides  detailed  information  on  the  characteristics  of  block  and 
lesson  tests.  The  TIE  is  intended  to  support  evaluations  of  test 
reliability  and  validity.  The  TIE  shall  include  at  least  the  following 
items  of  information: 

0  The  numbers  (or  percentages)  of  students  selecting  each  alternative 
answer  to  test  questions; 

0  Percentages  of  students  answering  questions  correctly; 

0  Average  test  score  of  students  answering  each  question  correctly; 

0  Flagging  test  questions  missed  by  over  70  percent  or  by  none  of 
the  students; 

0  I  tern- remainder  correlations  for  each  question; 

0  Means  and  standard  deviations  of  scores  for  each  objective  and  for 
each  full  test; 

0  Distribution  of  scores  on  the  full  test; 

0  Alpha  reliability  coefficients  for  each  objective  and  for  the 
full  test;  and 

0  Criterion-referenced  reliability  coefficients  for  each  objective 
and  for  the  full  test. 

The  TIE  shall  be  a  batch  mode  report.  An  on-line  program  shall  be 
provided  to  submit  the  report  requests  for  off-shift  processing. 

3. 7.'j  CAI  Evaluation  Data 

Detailed  quality  control  and  evaluation  data  shall  be 
collected  by  the  CAI  system.  This  information  shall  include  at  least 
the  following  items: 

0  Percentages  of  students  selecting  each  alternative  on  each 

question,  by  attempt  number  (first  attempt,  second  attempt,  etc.); 

0  The  means  and  standard  deviations  of  times  to  respond  to  questions 
and  times  to  read  text  passages; 

0  lists  of  unanticipated  responses  to  constructed  response 
questions; 

0  Lists  of  open-ended  student  comments  regarding  the  CAI  lessons; 
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0  Detailed  records  of  students'  paths  through  CAI  lessons;  and 

0  Suninary  statistics  on  major  units  of  CAI  lessons,  such  as  times 
to  complete  and  cumulative  scores. 

This  information  from  CAI  lessons  shall  be  available  to  authorized 
requesters  in  batch  mode  reports.  An  on-line  program  shall  be  provided 
to  submit  requests  for  off-shift  processing  of  these  reports. 

3.7.6  Course  Development  Management  Reports 

The  program  or  programs  provided  for  use  in  developing  on¬ 
line  and  off-line  tests  and  materials  shall  also  capture  and  store  in¬ 
formation  regarding  an  author's  work  using  the  editors,  and  the  status 
of  the  associated  course  material  development  effort.  This  information 
shall  be  summarized  in  management  reports  for  monitoring  courseware 
development,  production,  review,  revision,  and  implementation.  The 
reports  shall  include  such  information  as  identification  of  the  author 
responsible  for  developing,  reviewing  or  revising  a  lesson,  how  much  time 
has  been  expended  on  a  particular  task,  how  much  of  the  task  has  been 
completed,  and  the  expected  (i.e.,  scheduled)  completion  date. 

3.7.7  Performance  Forecast  Reporting 

The  system  shall  include  a  report  which  describes  a  student's 
performance  levels  in  completed  technical  training.  This  information 
is  intended  principally  for  a  student's  supervisor  in  a  post-training 
duty  assignment,  and  for  Field  Training  Detachment  (FTD)  or  On-the-Job- 
Training  (OJT)  managers  who  need  reports  of  individual  student  perfor¬ 
mances  (e.g.,  scores  and  times  to  complete)  in  resident  technical  train¬ 
ing  or  other  in-service  training. 

3.7.8  Special  Purpose  Reports 

Because  standard  reports  will  not  always  satisfy  all  manage¬ 
ment  and  administration  requi rentents ,  the  system  shall  include  an  open- 
ended  information  retrieval  system  that  can  be  used  to  answer  ad  hoc 
queries  regarding  students,  materials,  and  training  effectiveness.  The 
user  shall  be  able  to  specify  the  variables  which  he  wishes  to  examine, 
e.g.,  types  of  students  or  courses  by  specified  time  periods.  The 
output  shall  consist  of  the  number  of  pertinent  cases  identified  and  the 
mean,  standard  deviation,  and  range  for  each  included  variable.  This 
basic  data  retrieval  program  shall  also  be  interfaced  to  one  of  the 
standard  packages  of  statistical  programs  such  as  the  Statistical  Package 
For  The  Social  Sciences  (SPSS)  or  the  BMD  Biomedical  Computer  Programs, 
to  permit  extensive  analysis  of  student  data.  This  data  retrieval/ 
analysis  capability  shall  be  a  batch  mode  function.  An  on-line  program 
shall  be  provided,  however,  for  submitting  requests  for  off-shift  pro¬ 
cessing. 
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4.0  HARDWARE  REQUIREMENTS 


A  computer  hardware  system  shall  be  provided  to  support  the  instruc¬ 
tional,  administrative,  and  management  functions  required  by  Section  3.0 
of  this  specification.  The  system  shall  consist  of  comnercial ly  avail¬ 
able  equipment  with  the  capability  for  handling  the  types  and  numbers 
of  jobs  specified  in  Section  3.0.  The  hardware  components  provided  for 
this  system  shall  meet  the  general  intent  of  Military  Specification 
MIL-H-46855,  Military  Standard  MIL-STD-785A,  and  of  the  other  documents 
listed  in  Section  2.0. 

The  computer  system  shall  provide  a  capability  for  modular  buildup 
to  the  terminal  configuration  necessary  to  meet  the  functional  require¬ 
ments  and  assumptions  in  Section  3.0  of  this  specification.  The  system 
shall  operate  on-line  for  two  eight-hour  shifts,  five  days  per  week. 

Batch  operations  will  be  carried  cut  during  the  remaining  hours  or 
dun  ng  weekends . 

4.1  MAINFRAME  AND  PERIPHERALS 

The  central  computer  system  (i.e.,  mainframe  and  peripherals  in¬ 
cluding  any  communications  network  processing  unit)  shall  be  clustered 
and  housed  in  a  room  as  centrally  located  among  classroom  buildings  as 
is  practical.  This  room  shall  be  a  central  point  for  printing  CHI 
records  and  administrative  reports  anu  for  performing  software  main¬ 
tenance. 

The  central  system  shall  represent  the  minimum  configuration  con¬ 
sistent  with  meeting  the  requirements  of  this  specification.  Hiyh  speed 
communication  between  two  or  more  processors  of  the  sane  make  and  model 
shall  be  possible,  to  allow  for  future  expansions  that  might  require 
multiple  central  processors.  Consideration  shall  be  given  to  selecting 
a  computer  that  does  r.ot  require  such  special  environmental  conditions 
as  chilled  water  and  humdi fication.  Sufficient  air  concitioniny  and  a 
raised  floor  shall  be  installed  with  tne  system. 

4.1.1  Central  Processor 

The  processor  shall  be  capable  of  simultaneously  supporting 
at  least  the  number  of  student,  management,  and  administrative  terminals 
or  other  devices  needed  to  meet  the  functional  requirements  of  Section 
3.0  above.  To  support  routine  software  operations,  simple  on-line  source 
program  editing  operations  such  as  line  insertions,  deletions,  and 
replacements  shall  be  accomplished  in  an  average  response  time  of  not 
more  than  one  (  1)  second,  and  more  complex  on-line  operations  such  as 
file  1/0  shall  be  accomplished  in  an  average  response  time  of  not  more 
than  three  (3)  seconds. 

An  instruction  set  supporting  at  least  integer  and  floating  point 
ari thmetic,  string,  character  addressing,  branching,  logical,  looping. 
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and  assignment  operations  shall  be  provided.  Instructions  or  groups 
of  instructions  shall  also  be  availabl  to  implement  array  subscripting 
and  record  addressing. 

At  least  the  following  peripheral  devices  shall  be  attachable  to 
the  central  processor:  high  speed  mass  storage  devices*  off-line  storage 
devices  (e.g.,  tape  drives),  line  printers,  and  an  operator  console.  It 
shall  be  possible  to  add  central  memory  expansion  units  to  the  processor 
without  degrading  the  services  offeree  by  the  operational  system. 

A  capability  for  validating  aata  following  transfers  shall  exist, 
to  insure  that  transfers  are  successfully  and  correctly  accomplished. 

The  processor  shall  be  capable  of  supporting  batch  processing,  out 
not  necessarily  w-iile  supporting  on-line  CAI  and  CMI  operation. 

4.1.2  Central  Memory 

Sufficient  central  memory  shall  be  provided  to  support  the 
terminals  and  produce  the  response  times  dictated  by  the  functional 
requirements  of  Section  3.0  above  and  by  Section  4.1.1.  Any  type  of 
central  memory  (e.g.,  core,  MOS,  bipolar)  shall  be  acceptable,  provided 
the  required  response  times  and  validation  requirements  are  satisfied. 

4.1.3  Mass  Storage 

Mass  storage  shall  be  provided  to  support  the  instructional 
programs  and  data  required  by  Section  3.0,  and  the  system  programs  and 
data.  There  shall  be  at  least  two  mass  storage  devices,  to  provide  backup 
in  the  event  one  device  malfunctions.  The  mass  storage  transfer  rate 
plus  overhead  shall  be  sufficient  to  accomplish  the  required  response 
times. 

Channels  attached  to  the  mass  storaye  devices  shall  be  capable  of 
accomplishing  input/output  (I/O)  transfers  at  sufficient  rates  to  fulfill 
the  response  time  requirements  of  Sections  3.0  and  4.1.1. 

4.1.4  Off-line  Storage 

Off-line  storage  shall  be  provided  to  support  the  functional 
requirements  of  Section  3.0.  For  backup  and  reliability  purposes,  this 
storage  shall  be  accessible  via  at  least  two  off-line  storage  devices. 

The  off-line  units  shall  be  accessible  for  backing  up  information 
from  on-line  storage  devices  and  for  recovering  information  which  has  been 
temporarily  stored  off-line  to  lessen  the  on-line  mass  storage  require¬ 
ments.  Off-line  storage  shall  be  sufficiently  accessible  to  fulfill  the 
requirements  for  recovering  archived  CAI  lessons  (Section  3.5.2  above). 
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4.1.5  Central  Site  Printing 


Capability  shall  be  provided  for  printing  such  information 
as  classroom  rosters,  course  evaluation  summaries,  test  item  evaluation 
summaries,  selected  student  information,  CAI  reports,  on-line  and  off¬ 
line  course  materials,  software  programs,  and  computer  maintenance 
information.  One  or  more  central  site  line  printers  capable  of  printing 
the  128-character  ASCII  set  shall  be  provided.  With  an  even  distribu¬ 
tion  o*  all  characters,  a  minimum  rate  of  500  llnes/mlnute  (123 
characters  per  line)  is  required. 

4.1.6  Computer  Monitoring 

A  method  and  equipment  shall  be  provided  to  furnish  computer 
status  and  peripheral  device  Information,  initiate  diagnostics,  and 
generally  monitor  and  control  the  operation  of  the  entire  system. 

4.2  COMMUNICATION  SYSTEM 

Communication  hardware  and  software  sufficient  to  allow  remote  or 
local  operation  of  the  terminal  system  specified  in  Section  4.3  below 
shall  be  provided.  Standard  protocols,  electrical  interfaces,  and  data 
rates  shall  be  used  throughout.  This  communication  system  shall  be 
capable  of  supporting  the  mix  of  terminal  capabilities  specified  in 
Section  4.3  and  the  response  requl rements  of  Section  3.0  of  this 
specification.  The  communi cation  system  shall  be  modularly  expandable 
beyond  the  minimum  number  of  terminals  required  in  this  specification 
even  though  the  software  and  hardware  system  is  initially  configured 
for  that  number  of  terminals. 

4.2.1  Computer/Network  Interface 

The  network  processing  hardware  at  the  computer  site  shall 
be  able  to  transmit  data  to  and  receive  data  from  the  total  number  of 
terminals  required  by  this  specification,  and  the  electronics  shall  be 
modularly  expandable  to  handle  additional  terminals. 

The  computer  electronic  ports  for  system  terminals  shall  meet 
industry-wide  standard  interfacing  requirements.  The  network  processing 
equipment  shall  have  the  capability  of  detecting  and  correcting  trans¬ 
fer  errors  on  characters  and  control  codes  received  from  the  terminals. 

For  transmitting  data  between  the  computer  and  terminals  at 
locations  remote  from  the  central  site,  the  most  cost  effective  method 
shall  be  provided. 

4.2.2  Network 

The  network  shall  consist  of  dedicated  voice  grade  lines, 
configured  to  provide  the  data  rates  needed  to  satisfy  the  functional 
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requirements  of  this  specification. 

4.3  TERMINAL  SYSTEM 

The  terminal  system  shall  be  composed  of  a  mix  of  types  of 
terminals  and  other  devices  that  includes: 

(1)  alphanumeric,  graphics,  and  color  display  types  of  terminals 
for  students  and  administrative  use;  and 

(2)  management  1/0  device:,  to  provide  the  required  CHI  functional 
capahi 1 i ties . 

This  compatible  set  of  devices  shall  be  located  in  or  near  class¬ 
rooms,  and  shall  he  capable  of  remote  operation  with  the  central 
computer. 


4.3.1  General  Characteristics 

All  display  terminals  selected  shall  have  the  electronic, 
display,  and  keyboard  characteristics  as  listed  in  Sections  4. 3. 1.1, 

4. 3. 1.2.,  and  4. 3. 1.3  below.  Management  devices  shall  conform  to  Section 
4. 3. 1.1. 


4. 3. 1.1  Electronic  Characteristics.  Each  terminal  shall  be 
capable  of  sending  and  receiving  data  at  selectable  i ndus try-standard 
rates.  Transmission  shall  be  via  an  industry-standarJ  interface. 
Terminals  or  associated  communications  equipment  shall  have  at  least 
the  capability  of  detecting  validation  errors  on  single  incoming  alpha¬ 
numeric  and  control  characters,  with  all  control  codes  and  character 
codes  conforming  to  industry  standards.  Any  terminal  shall  be 
electronical ly  i  nterchangeable  with  anv  other  at  any  communications 
port. 


4. 3. 1.2  Oisolay  Characteristics.  Each  display  terminal 
shall  provide  at  least  24  lines  of  display,  with  64  characters  or  inore 
per  line,  on  a  screen  of  at  least  12  inches  diagonal  measurement,  The 
96  character  ASCII  set,  consisting  of  upper  and  lower  case  alphabet, 
numbers,  and  punctuation  shall  be  available.  The  cursor  shall  be 
controllable  so  that  it  can  be  located  to  any  position  on  the  screen. 
BuIk  erase  capability  shall  be  provided. 

4. 3. 1.3  Ke^boarc  Characteristics .  The  keyboard  on  each 
display  terminal  shall  be  in  the  standard  QWERTY  format  for  alphabetic 
characters,  with  numbers  anti  punctuation  keys  also  available.  All  keys 
must  be  plainly  and  permanently  marked  to  identify  characters  and/or 
functions  controlled. 
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4.3.2  Student  and  Administrative  Terminals 

A  mix  of  alphanumeric-only,  alphanumeric/graphics ,  and 
alphanumeric/graphics/color  terminals  shall  be  provided  for  student  and 
administrative  use.  The  numbers  of  terminals  of  the  various  types  shall 
be  sufficient  to  provide  the  tunctional  capabilities  described  in  Section 
3.0,  and  shall  meet  any  additional  requirements  imposed  by  character¬ 
istics  of  the  Air  Force-designated  courses  to  be  implemented  on  the 
system.  Ar,  external  jack  for  controlling  a  device  such  as  a  slide  pro¬ 
jector  shall  be  provided,  if  course  characteristics  indicate  that  such 
a  device  is  instructional  ly  desirable.  The  resolution  provided  by 
graphics  terminals  shall  be  sufficient  to  accomodate  the  course 
materials  to  be  implemented.  Color  terminals  shall  provide  at  least 
eight  different  colors,  including  black  and  white,  with  color  available 
for  foreground  letters  and  figures  and  for  background.  An  option  to 
fill  colors  within  graphics  polygon  boundaries  is  desirable.  In  addition 
to  the  keyboards  provided  for  student  and  instructor  interactions,  the 
system  shall  also  support  a  device  such  as  a  light  pen  or  touch  panel, 
to  facilitate  interaction  with  graphics  displays.  To  facilitate  pre¬ 
paration  of  graphics  displays,  the  system  shall  support  a  digitizing 
tablet  or  equivalent  device. 

4.3.3  Management  Devices 

Management  devices  in  types  arid  r.umt.rs  sufficient  to  satisfy 
the  CMI  requirements  described  in  Section  3.C  shall  be  provided.  As 
necessary  to  meet  the  instructional  and  administrative  needs,  these  shall 
include  devices  to  read  forms,  to  print  student  assignments  and  class¬ 
room  operations  reports  (such  as  rosters,  absence  reports,  and  regis¬ 
tration  data),  and  such  other  devices  as  may  be  required  and  cost 
effective  for  the  designated  installation  site. 

4.4  RELIABILITY 

An  effective  reliability  program  shall  be  established  and  maintained, 
within  the  general  intent  of  MIL-STD* 785A.  This  program  shall  include 
failure  reporting  and  design  changes  to  correct  pattern  failures.  The 
program  shall  be  adjusted  to  suit  the  types  of  procurement  and  equip¬ 
ment,  and  shall  be  directed  toward  maximizing  the  mean  time  between 
failures  (MT8F)  for  individual  equipment  items.  System  reliability  shall 
be  sufficient  to  meet  the  availability  requirements  of  Section  4.6. 

4.5  MAINTAINABILITY 

A  maintainability  program  snail  be  established,  within  the  general 
Intent  of  MIl-STD-470  This  program  shall  incorporate  a  maintenance 
concept,  intended  to  minimize  the  mean  time  to  restore  (MTTR).  System 
MTTR  shall  be  sufficient  to  meet  the  avai lability  requirements  of 
Section  4.6. 
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4.6  AVAILABILITY 


System  availability  shall  be  defined  as  follows: 

System  Intrinsic  Availability  -  KT3F  /  (MTBF  +  MTTR) 

where  MTBF  is  defined  as  the  system  mean  time  between  failures  (i.e., 
the  MTBF  resulting  from  combining  the  MTBFs  of  all  the  individual  equip¬ 
ment  items),  and  MTTR  is  defined  as  the  mean  time  to  restore  the  system 
after  a  failure  to  any  equipment  in  the  system. 

The  System  Intrinsic  Availability  for  the  low-cost  CAI/CMI  system 
shall  be  at  least  0.98. 


'V’IS  COVMMHI,?  nhMIiNG  OFftCl' .1  gs-i. 


152 


